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Assessing Organizational Effectiveness: 
Considerations and Procedures 



Section 1 
Introduction 

Sitting in his spartan little office, the NCHEMS researcher 
looks painfully at his ringing telephone. He is certain rhis is 
another administrator looking for insight, if not answers, 
concerning what should be assesRsd to determine if their 
organization is effective. The researcher recalls what happened 
the last time he tried to respond to such a question. 

He started talking about the complexity of problem, about the 
role different constituencies play in determining the focus, and 
about how the same data can be used to inform many different 
questions, and ... it took about five minutes. The president 
broke in and said, "Look, I don't want to deal with all that. I 
just want to assess our effectiveness." He had learned not to 
start over. He understood that practitioners want answers, and 
that they don't have time and/or want to hear about the complexity 
of the problem. Unfortunately, neither the notion of 
effectiveness, nor its assessment is simple. 

This study is concerned with identifying the dimensions of 
effectiveness that are pertinent to postsecondary institutions, 
and the means of assessing them. The reader will find information 
about criteria that can be employed to assess effectiveness. 
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reasons for choosing them, standardized measures and indicators 
for assessing them, and other factors critical to assessment. 

For better or worse, this work will not provide a 
prescription or formula that, if followed, guarantees successful 
assessment. Rather, it is more like a tourist's guide-book that 
describes where you might want to go, and provides a map for 
determining the many ways you can get there. However, the 
decision as to where you go, and how you get there remains in your 
hands. The two things that you can be sure of are that (1) you 
are not traveling alone, and (2) whatever you decide, you are not 
going to satisfy everyone traveling with you. 

Effective Organizations and Effective Automobiles 

The term effectiveness is used so often, surely we all must 
know what it means? In fact, this may be the case, but this is 
not to suggest that our approaches to defining effectiveness are 
necessarily shared. Just as most western religions were borne of 
different interpretations of the Bible, effectiveness has many 
different minions, some of which are in conflict. The complexity 
of the construct and the problems associated with it can be 
illustrated by developing an analogy between evaluating 
effectiveness at the organizational level and evaluating the 
effectiveness of an automobile. 1 



Iwe recognize that people seldom, if ever, talk about the 
effectiveness of an automobile. However, we believe that the 
comparison provides insight into the problem at hand. 

-2- 



1 « 

Ieric 



For example, first we establish the "point of view." That 
is, the concerns of a typical consumer are likely to be very 
different from those of w.. automobile manufacturer, a loan officer 
at a bank, an automobile mechanic, or an environmentalist. 
Subsequent considerations will be a function of the needs, 
preferences, and values of whoever 's point of view is being 
considered. 

Similarly, in a university, there are many constituents 'vhose 
points of view differ. One example is class scheduling. Students 
may prefer midday classes, while teachers find morning hours 
ideal. Administrators, on the other hand, may prefer to schedule 
classes throughout the day. Effectiveness is based on meeting 
preferences of the individual constituent groups, but some of the 
preferences clearly conflict. 

Another dimension of evaluating effectiveness identifies the 
criteria to be employed. For example, some consumers may 
primarily be concerned about how an automobile looks, what it says 
about their status, state of the art technology, or how fast a car 
can accelerate from zero to sixty miles per hour. Others may be 
concerned about cost, efficiency, maintenance, resale value, or 
ability to operate under winter or mountainous conditions. If the 
consumer is a tradesman, he/she may be concerned about such things 
as structure, load capacity, and the ability to meet the needs of 
a growing business. 

An educational organization also may attract members who 
reflect differences in establishing criteria of effectiveness. 
Some students may enroll to satisfy a concern for status, while 
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others s.ek low tuition, others may be attracted because the 
organization offers a particular course of study or set of 
faculty. All of these examples of student criteria suggest that 
effectiveness will be differentially applied based on what the 
educational consumer considers important. 

Third, once a decision has been made on what criteria are 
pertinent, we must determine how to measure them. For example, if 
we are concerned about efficiency, should we assess cost in terms 
of the sticker price of the car; amortized costs ever three, five, 
or eight years; and/or include maintenance costs. We note that 
economists actually distinguish between five different types of 
efficiency: production, technical, price, preference, and 
exchange . 

Similarly, if efficiency in higher education is believed to 
be a high quality education at a low cost, how should we measure 
these factors? Quality can be measured by faculty interest in 
education, source and level of faculty degrees, student contact 
hours, or small class size (to name a few). Does least cost 
include tuition, or are there other costs that should be 
considered such as fees, housing, or opportunities lost? 

Having made these decisions, we must recognize that there are 
probably criteria that members of each of these consumer groups 
should have employed that were ignored. Consumers primarily 
concerned with the extent to which a vehicle meets their status or 
image-needs may find themselves in a vehicle that looks terrific, 
but is structurally unsound, or, that is simply too expensive to 
maintain. The consumer primarily concerned with cost and 
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efficiency may find that his spouse is about to leave him because 
she was not consulted. The tradesman may find a given vehicle 
ideally suited to the demands of his job, but discovers that 
he/she can't afford to make the payments when business is slow. 

The same set of multiple, implicit criteria for assessing 
effectiveness are also applicable to higher education. Faculty 
Whose main interests are teaching may also experience pressure to 
publish, finding that the two activities are not always 
complementary. The issue then becomes one of how to balance 
differing requirements for becoming effective. 

In summary, this analogy suggests that all of the multiple 
and competing dimensions of effectiveness that contribute to 
evaluating ownership of an automobile are also present in 
organizations of higher education. However, as these examples 
suggest, individual consumer decisions pertinent to auto ownership 
are multiplied at the organizational level to include many more 
participants and criteria for evaluating effectiveness. 

The following work suggests that managing the assessment of 
effectiveness in organizations begins with understanding that 
definitions of effectiveness depend on the evaluator's point of 
view. Moreover, criteria for assessing effectiveness depend not 
only on point of view, but also on multiple and competing 
individual preferences and choices. Finally, although 
organizational managers may be able to identify multiple poir.ts of 
view and the criteria applicable to assessing effectiveness based 
on chose points of view, an additional issue is how to measure 
effectiveness. 
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Issues pertinent to assessing effectiveness are the focus of 
this work. Accordingly, subsequent sections describe how 
effectiveness has been viewed, isolate four evaluative domains of 
effectiveness based on existing views, and use those four domains 
and their criteria to categorize and identify instruments that 
measure many dimensions of effectiveness in higher education 
organizations. 
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Section 2 

Effectiveness, Excellence, Quality, and Performance 

Effectiveness has been equated with quality and excellence. 
Some have gone so far as to equate it with something so banal as 
performance. "Performance," the dictionary (Webster, 1981) 
informs us, is the "execution of an action; something 
accomplished." Certainly quality and excellence must entail 
something more than simple action? Can effectiveness really 
encompass such broad interpretations? In hopes of answering these 
questions we shall provide a brief review of research that 
addresses, and hence may lend insight into, effectiveness, 
quality, excellence, and performance in postsecondary education. 

A dictionary provides information about the meanings and 
interpretations of particular subjects. We could argue that the 
ERIC (Educational Resources Information Center) database can be 
used for the same purpose. That is, one can use the database to 
identify all the research related to a particular subject (for 
example, organizational effectiveness) and thereby gain insight 
into its interpretations and meanings. 

A search of the ERIC database for the years between 1966 and 
1985 yields more than 1200 entries for the descriptor 
"organizational effectiveness," Restricting the search to a 
combination oi! "organizational effectiveness" and "colleges" 
produces more than 300 entries. The articles identified in the 
restricted search focus on topics that include: public policy and 
college management; outcomes research; student involvement, 
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learning, and development; assessing social and economic benefits 
to the community; relationships between state colleges and state 
and local agencies; structuring the institution for improvad 
effectiveness; power, motivation, leadership style, and managerial 
effectiveness; collective bargaining; administrative malpractice; 
management by objectives; increasing productivity; renewal; 
organizational development; innovation. 



Expert Opinion 

Richard E. Peterson developed both the Institutional 
Functioning Inventory (Educational Testing Service), and the 
Institutional Goals Inventory (Educational Testing Service). 
These instruments have been used by hundreds of institutions as a 
means of gathering information that could be used for 
institutional planning, self-study, accreditation, and evaluation. 
In articles discussing the assessment of effectiveness, Peterson 
(1971, p. 15) writes, 

I would like to think that "effectiveness" and "quality" 
mean the same, that an institution that is effective 
according to one or more of the criteria to be outlined 
[student learning; student values development; programmatic 
responsiveness; campus morale], also has quality or is of 
high quality. 

Peterson and Vale (1973, p. 1) reiterate this theme: 

How might institutional "effectiveness" or institutional 
"quality" be defined? In a recent special report, the 
Carnegie Commission on Higher Education (1973) stressed the 
notion of "value added:" 
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The quality of an institution should be determined by 
what it does for the students it enrolls, not by tie 
characteristics of its entering students ... (p. 39 '. 

Thus, the fundanental index of institutional 
quality/effectiveness for most colleges and universities 
would be how much the student learns or otherwise develo. s as 
a result of attending the institution. . .Effectiveness , as t 
is discussed here, is defined fonly) with reference to goa 
institutional effectiveness (quality) mean in short, 
achievement of institutional goals. 

Richard I. Miller is a consultant of national reputatio.. 
whose expertise is the assessment of institutional performance. 
In one of the many books he has written on assessing performance, 
he writes (1979, p 6) : 

In attempting to define quality, we can easily be distracted 
by the great diversity among post secondary institutions, 
forgetting all they have in common. For instance, all 
obviously have students, faculty members, administrators, 
alumni. They share certain curricular f eatures. . . All are 
guided by some policies and procedures; all have some 
politics to cope with; all have some type of 
evaluation. . .And of course they share one primary goal — 
educating students. 

He goes on to argue that while the goals, policies, and 
programs* and so on, are arranged uniquely in each institution, 
"high quality" colleges and universities also have some more 
specific characteristics in common: 

Their members have reached general agreement on where they 
want to go, how their limited resources can be employed in 
making progress toward goals, and whether they are moving 
toward where they want to go. Their academic programs 
contribute to the students' ability to think clearly, 
communicate effectively, act wisely, disciiminate among 
values and progress toward career aspirations. Their faculty 
members are on the growing edge of tiieir academic fields, 
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seeking new ideas r materials r and approaches to improvement; 
they work closely with students in formal and informal 
teaching and learning circumstances; and they participate 
meaningfully in institutional governance. Their 
administrators exercise dynamic^ and sometimes courageous, 
leadership in helping to make crucial decisions about the 
institution's future, and they use human and material 
resources prudently in accordance with established goals and 
objectives. And finally, the trustees of high-quality 
colleges and universities support their institutions while 
serving as friendly critics, provide policy direction, 
differentiate between policy development and operational 
procedures, and evaluate institutional effectiveness (p. 6). 

Killer then argues that whether or not one agrees that a high 
quality institution would have all the characteristics cited 
above, it is clear that an assessment of its quality cannot be 
based on a single indicator or even a cluster of similar indices. 
He goes on to identify ten areas and 45 evaluative criteria that 
evaluators might examine in assessing institutional quality. 
These are sr.own in Table 1. 

Miller concludes his discussion of quality by noting that, 

"in the final analysis, the validity of any criteria of 

institutional quality is a matter of judgment." He then quotes 

from Allan Carter (1966, p. 4): 

Quality is an elusive attribute, not easily subjected to 
measurement .. .In the operational sense, quality is someone's 
subjective judgement, for there is no way of objectively 
measuring what is in essence an attribute of value. 

While Miller's and Carter's point regardira the subjective 
aspect of quality is not new, it is extremely important and a 
fundamental theme of this work. Every discussion or argument 
concerning what constitutes effectiveness, quality, and/or 
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Table 1 

Institutional Evaluation Criteria 
from Richard I. Miller, 1979, p. 7-10 

G oals and Objectives 

1. The goals statement serves as an effective guide for the 
present and future. 

2. Objectives reinforce goals. 

3. The institution has adequate planning capabilities. 

4. Institutional admissions policies and procedure*? are 
consistent with institutional goals and objectives. 

5. The institution's goals and objectives help it maintain a 
reasonable identity within a statewide system of 
institutions . 

Students ' Learning 

6. Students give a good rating to their advising and counseling 
system. 

7. Retention rates are reasonable. 

8. An array of individualized and compensatory learning resources 
is available. 

9. The student affairs administration is effective. 

10. Satisfactory progress is evident toward learning goals. 
Faculty Performance 

11. Current policies and procedures for evaluating individual 
faculty members a: satisfactory. 

12. Current instructional improvement/faculty development programs 
serve their purposes. 

13. Faculty personnel policies and procedures are considered 
satisfactory . 

14. Faculty salary sales and f i inge benefits are competitive. 

15. The overall quality of the taculty's performance is optimal. 
Academic Programs 

16. The institution has effective policies and procedures for 
developing new programs. 
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17, The institution h?s effective policies and procedures tor the 
review and evaluation of existing programs. 

18 • The general education component is an intellectually 
stimulating and integral part of the curriculum. 

19. The quality and size of the graduate program are consistent 
with institutional goals and objectives. 

20. The library or learning resources center provides good service 
to the academic community. 

Institutional Support; Services 

21. The physical plant and facilities are adequate for the size of 
the student body and for the nature of the academic program. 

22. The institution has: a relevant and current long range plan for 
developing and maintaining its physical plant. 

23. Salaries and other benefits for support personnel are 
sufficient to attract and retain competent indi^riduals. 

24. Systematic procedures are used for evaluating the performance 
of support personnel. 

Administrative Leadership 

25. The administration gives adequate attention to planning. 

26. The chief campus administrator and his team have effective 
working relationships with other campus administrators. 

27. Institutional governance policies and procedures allow for 
effective institutional management. 

28. The policies and procedures established for administrative 
evaluation and for professional development are satisfactory. 

29. The institution has an effective affirmative action program. 
Financial Manag em ent 

30. The tuition and fee structure is compatible with the 
institution's needs and with the students' capacity to pay. 

31. The institution has an efficient management system for 
accounting and financial reporting. 

32 • Costs and expenditures are comparable with benchmark 
institutions. 
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33. The investment portfolio is well managed. 

34. The institution has an effective system for demonstrating its 
accountability . 

Governing Board 

35. The policies and procedures for conducting board affairs are 
satisfactory , 

36. Trustees understand the differences between policy formulation 
and policy implementation and apply this knowledge. 

37. The governing board works effectively with external 
constituencies. 

38. The board contributes positively to improving the 
institution. 

External Relations 

39. The institution's activities contribute to the quality of life 
in its primary service area. 

40. The institution has effective relationships with the state 
higher education (coordinating or governing) office. 

41. The institution has an effective relationship with the federal 
government. 

42. The institution is able to secure acceptable levels of funding 
from private sources and foundations. 

Institutional Self-Improvement 

43. The institution seeks improvement through innovation and 
experimentation . 

44. Campus groups have positive attitudes toward self- 
improvement. 

45. The institution has established procedures for evaluating its 
oWi^ effectiveness. 
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excellence is based on someone's values. We shall return to this 

point many times in the discussion that follow. 

Peter Ewell is a recognized authority in the field of higher 

education on the subject of educational outcomes. Hence, it is no 

surprisa that when he talks about "excellence" in postsecondary 

education his judgments rest on the achievement of educational 

outcomes. His "prejudice," at least from this writer's 

perspective, is eloquently expressed by an excerpt from his book. 

Dimensio ns of Excellence in Postsecondary Education (1985, p. 1): 

On the one hand, excellence is judged in terms of particular 
sets of standards — available resources, the structure of 
programs and curricula, and the intellectual characteristics 
and attainment of faculty and students. This has been a 
theme most particularly evident in the process of 
postsecondary accreditation. On the other hand, these sets 
of standards are taken to be indicative of an underlying 
process. High levels of resources, well-structured programs, 
and an able faculty and student body are assume'1 to be highly 
correlated with educational growth. They are largely valued 
because of the perceived increments of educational growth 
they are believed to produce. 

The argument will be made that institutions achieve 
"excellence" in postsecondary education in so far as they 
produce demonstrable changes along dimensions consistent with 
(1) institutional objectives, (2) student educational growth, 
and (3) the expressed needs of society and particular 
constituencies within society. 

Reputational Ratings 

The desire for national recognition as a "quality" 
institution or department influences the decisions made in a 
number of colleges. One measure of quality that is seriously 
attended to by many colleges and departments are reputational 
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■ ratings. The examination of how such indices are generated mayr 
accordinglyr further enlighten us on the attributes of a quality 

I institution. 

^ In an article entitled^ "Are Reputational Ratings Needed To 

0 Determine Quality r" Astin and Solomon (1981) conclude that 

■ measures of prestige and selectivity proved to have substantial 
correlations with mean ratings of the "overall quality" of 

1 undergraduate education. Quality, for purposes of this study, was 
defined in terms of peer ratings on six dimensions: (1) the 

■ scholarly and professional accomplishments of the faculty, (2) 

■ faculty commitment to undergraduate teaching, (3) overall quality 
of the undergraduate edr-ation in their fields, (4) preparation of 

I students for graduate or professional schools, (5) preparation of 

students for employment after college, and (6) innovativeness of 

■ curriculum and pedagogy. 

Accreditation 

■ Reason would suggest that any institution which is 
"effective," must almost certainly be "accredited." (However, we 

I are not prepared to argue that the reverse is true.) Acceptance 

» of this premise suggests that we might gain further insight into 

what constitutes institutional effectiveness by reviewing what is 
I evaluated in order to c^^-n accreditation. Richard M. Miller 

(1983, p. 35) argues that, 

I The crucial question for accreditation is not how one 

institution or program compares with another but how 

■ effectively the institution or program meets its own 

■ educational objectives .. .What accreditation attests to is 
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that an institution or program has clearly defined and 
educationally appropriate objectives r that it maintains 
conditions under which their achievement can be reasonably 
expected r that it appears in fact to be accomplishing them 
substantially^ and that it can be expected to continue to do 
so. 

Robert Chambers (1984) r offers the following list of areas 
(taken from criteria established by the Middle States Association 
of Colleges and Schools) as examples of what is examined in 
judging whether an institution should be accredited: 

1. The institution's mission, goals, and objectives; 

2. Its program and curricula for fulfilling these; 

3. The "outcomes" it achieves (its current effectiveness) 
and hopes to achieve (its aspirations); 

4. Its admissions policies, efforts, and results; 

5. The services it provides its students once they are on 
campus; 

6. The role of its faculty in serving a school's stated 
mission and meeting its students' needs; 

7. Its administrative organization for facilitating 
teaching and learning; 

8. The role of its governing board (trustees) in ensuring 
that the announced mission and goals are fulfilled; 

9. Its accounting, budgeting, and accounting policies 
and practices; 

10. Its library, and other learning resources, including 
computings role in meeting the published objectives; 

11. The adequacy of its plant and equipment; 

12. Its catalogueF, other publications, and general public 
relations posture; 

13. Its openness to innovation, experimentation, and 
future growth. 
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■ Conclusion 

A colleague reviewed a draft of the preceeding materials* He 
I offered a conunent and a question. Both dealt with the attempt to 

demonstrate that it is extremely difficult to differentiate 

■ between effectiveness, quality, excellence, and performance. The 

■ comment is probably best left unrepeated. However, it may be 
useful to pursue his question. 

■ He remarked, "Are you arguing that there is no difference 
between an "effective" institution, and a high "quality" 

m institution? If this were true, and I don't believe it is, then 

m there are a great many students paying for an Ivy League education 

when they might do just as well at their local state college." 
I Our response was yes, and no. Ii your criterion of 

effectiveness is achievement of objectives then a state college 
g might be just as effective as an Ivy League school. The state 

g college might even be more effective. Certainly, there are other 

■ effectiveness criteria one could employ such that the state 
I college came out at the bottom of the pile — for example, 

selectivity, library resources, or research grants. 
I However, irrespective of whether one is talking about 

effectiveness, quality, or excellence, the criteria and standard 

■ used to judge them will be subjective. More precisely, both tht 
I focus and judgment of effectiveness is a function of individual 

needs, prel'erences, and values. If I want to be trained as an 
I auto mechanic, then an Ivy League school is worthless to me. If I 

want to be a scholar, then I must go to an institution that 

■ supports and rewards such efforts. If I am a legislator 
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confronted with the decision of providing financial support to the 
Ivy League or the state school, then I better know what my 
constituents want and are willing to nay for. 

Hopefully, the previous discussion has shown that if three 
people were asked to describe how they would judge whether an 
organization was effective, they, like the blind men asked to 
describe an elephant, might appropriately note very different 
characteristics. The discussion that follows prov.-ides a framework 
for understanding and evaluating most of the criteria that have 
been discussed with respect to organizational effectiveness. 



r o 

ERIC 



-18- 

22 



Section 3 
Organizational Effectiveness 



One might forgive the literature in higher education for 

■ being imprecise with respect to its treatment of effectiveness. 

■ Unfortunately^ the perspective from organizational theory, a 
domain th^t willingly takes credit for inventing the construct r 

■ provides little practical insight or guidance for dealing with 
effectiveness. The literature that flows from organizational 

I theory seems primarily concerned with three issues. First , to 

M describe why the problem is so complicated. Second , to explain 

■ why previous attempts to deal with the problem have been 

■ unsatisfactory. And, third, to offer unique alternatives that 
inevitably suffer from the same type of problems that these 

I alternatives purport to solve. 

— Most articles on organizational effectiveness written in the 

■ last ten years begin with a review and criticism of two views of 

■ effectiveness: goal centered and natural systems. Goal centered 
views (Etzioni, 1964; Georgopolous and Tannenbaum, 1957; Hall, 

g 1978; Price, 1968), which include such variants as Management By 

Objectives (Odiorne, 1965; Humble, 1970) and Cost/Benefit Analysis 
I (Hitch, 1965; Rivlin, 1971), assume: 

■ ...che organization is in the hands of a rational set of 

■ decision makers who have a set of goals in mind which they 
wish to pursue. Further, these goals are few enough in 

■ number to be manageable and can be defined well enough to be 
understood. Given that the goals can be thus identified it 

^ should be possible to plan the best management strategies for 

H attaining them. Within this orientation the way to assess 
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organizational effectiveness would be to develop criterion 
measures to assess how well the goals are being achieved. ^ 

Goal centered models are usually criticized on the grounds 
that there is a wide difference between espoused public goals and 
the goals that are in fact pursued; that organizations often have 
multipler contradictory goals; that in many cases goals must not 
be reached or the acquisition of resources would no longer be 
justified; that goals are often post hoc justifications of what is 
already going on (Faerman & Quinn, 1985^ p- 1). 

The natural systems viewr which incorporates such var:.ants as 

Che Systems Resource (Seashore and Yuchtman, 196V) and Open 

Systems (Etzionir 1964) models r make the assumption that: 

...if an organization is of any size at all the demands 
placed upon it are so dynamic and so complex that it is not 
possible to define a small number of organizational goals in 
any way that is meaningful. Rather, the organization adopts 
the overall goal of maintaining its viability or existence 
through time without depleting its environment or otherwise 
fouling its nest. Thus to assess an organization's 
effectiveness one shoud try to find out if an organization is 
internally consistent within itself, whether its resources 
are being judiciously distributed over a wide variety of 
coping mechanisms, whether its using up its resources faster 
than it should, and so forth. ^ 

A major criticism of this view is that it implicitly uses 
goals, even though the models which inco^rporate this view strongly 
criticize goal based models, in addition, models which 
incorporate this view seldom describe what appropriate coping 



-^John P. Campbell, David A. Bownas, Norman G. Petersen, and 
^ Marvin D. Dunnette. The Measurement of Organizational 

■ Effectiveness; A Review of Relevant Research and Opinion , 1974, p. 

™ 6 . 

2lbid, p. 6. 
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mechanisms are, or how to operationally assess efficiency and the 

judicious use of resources. 

Reviewing a score of articles dealing with the assessment of 

organizational effectiveness, Cameron (1978, p. 604) observes: 

Difficulty in assessing organizational effectiveness has 
arisen because no one ultimate criterion of effectiveness 
exists. Instead organizations may pursue multiple and often 
contradictory goals (Warner, 1968; Perrow, 1970? Hall, 1972 
1978; Dubin, 1976), relevant effectiveness criteria may 
change over the life cycle of an organization (Yuchtman and 
Seashore, 1967; Kimberly, 1976; Miles and Cameron, 1977), 
different constituencies may have different importance at one 
time or with regard to organizational aspects and not others 
(Friedlander and Pickle, 1968; Scott, 1977; Barney, 1978), 
criteria at one organization level may not be the same as 
those at another level (Price, 1972; Weick, 1977), and the 
relationships among various effectiveness levels may be 
difficult to discover (Seashore, Indik, and Georgopolous , 
1960; Mahoney and Weitzel, 1969; Kirchoff, 1975). In short, 
organizational effectiveness may be typified as being mutable 
(composed of different criteria at different life stages), 
comprehensive (including a multiplicity of dimensions), 
divergent (relating to different constituencies), 
transpositive (altering relevant criteria when different 
levels of analysis are used), and complex (having 
nonparsimonious relationships among dimensions). 

The problems associated with assessing organizational 
effectiveness niay be summarized by six questions (Cameron, 1978; 
Steers, 1977, Faerman and Quinn, 1985): 

(1) whose perspective should dominate the assessment of 
organizational effectiveness (e.g., organizational members, 
external resource providers, dominant coalitions, etc.); 

(2) what is the appropriate level of analysis (e.g., 
organization level, organizational subunits, individual, 
etc. ); 

(3) what are the appropriate types or sources of data (e.g., 
organizational records, perceptual measures, researcher 
observations, etc. ); 
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(4) what should be the appropriate time frame (i.e., long- 
term vs. short-term effectiveness); and 

(5) what is the referent against which effectiveness is to be 
assessed (i.e., is there a norm or standard, is the 
organization being compared to other organizations, to 
itself, etc.). 

The sixth and most obvious (but not necessarily critical) 
question that must be dealt with in assessing organizational 
effectiveness is identification of the domain of assessment. 
However, in our opinion, one cannot practically address this 
problem until we first resolve whose perspective or preferences we 
are concerned about. In other words, the criteria of what 
constitutes an effective organization, at least in higher 
education, are likely to substantially differ depending on whether 
one is talking to state legislators, students, faculty, 
administrators, members of the business community, accrediting 
agencies, and so forth. 

In the work that follows we will develop a paradigm that 
includes and discusses the aforementioned issues, criteria that 
may be used to assess effectiveness, and measures and indicators 
for assessing them. 
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Section 4 
An Effectiveness Paradigm 

The paradigm that follows rests on five premises. The first 
premise follows from the previous discussion concerning what 
constitutes effectiveness. The premise concerns the definition of 
effectiveness and its equation with performance. Webster (1981) 
defines "effective," the adjectival form of "ef f ectiveness r " as 
follows: 

la. producing a decided, decisive, or desired effect; 
lb. impressive, striking; 

2. ready for service or action 

3. actual 

4. being in effect; operative 

5. producing or capable of producing a result. 

We previously noted that some theorists equate effectiveness 

and performance. For example, Kanter and Brinkerhoff (1981, p. 

322), in a seminal review of the effectiveness literature write: 

...fo the purpose of this paper we make no hard-and-fast 
distinctions among effectiveness, productivity, performance, 
and succes«3. Rather, we look for all related measurement 
issues and use effectiveness and performance aa general and 
interchangeable terms. (At the same time we acknowledge the 
importart conceptual distinction between efficiency — "doing 
things right" — and effectiveness — "doing the right thing.") 
Performance (Webster, 1981) is defined -n: 

la. the execution of an action 

lb. something accomplished: deed, feat 

2. the fulfillment of a claim, promise, or request: 
implementation 
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■ The primary reason for suggesting that there is little 
difference between the notion of effectiveness and performance is 

I because it emphasizes that (1) the focus of (effectiveness) 

assessment may be wide ranging — as suggested ^ for example , in the 

■ previous discussion concerning the attributes of effective^ 

m excellent r and quality institutions; and (2) the focus is largely 

determined by key constituenci3S to the assessment process. 

■ However^ we r^uggest that performance in-and-of itself is 
esisentially neutral. Only when someone's values are considered 

I can a judgment be made regarding effectiveness. 

■ The second premise is that institutions of higher education 
may be viewed as being composed of the components and 

■ subcomponents shown in Figure 1. Not shown here, however , and 
critical to the institution are its constituents. These include, 

1 for e; nple, faculty, students, administrators, its community, 

2 ousiness, the state and federal government, etc. This premise 
serves two purposes. First, it forces us to recognize that the 

■ effectiveness of an institution may rest on judgments regarding 
any one or combination of its components. And second, that the 

I criteria suggested as appropriate for assessing overall 

^ institutional effectiveness should generally be applicable to 

■ assessing any one or more of the institution's comporents. 

■ The third premise follows from the previous two: the 
dimensions of effectiveness that are measured and how they are 

I measured are largely a function of who is ocking the questions and 

how they wish to use the data. This pretnise (]. ) reemphasizes the 

■ notion that organizational effectiveness is not limited in scope 
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to a single dimension or focus; and (2) that the focus is 
constituency driven • 

The fourth premise concerns the meaning of "criteria of 
effectiveness," "indicators of effectiveness," and the 
relationship between them. Webster (1981) defines a criterion as 
"a standard on which a judgment or decision may be based*" An 
indicator is defined as "a device that measures or records and 
visibly indicates." 

This suggests that a "criterion of effectiveness" is 
something that effectiveness is contingent on, or a function of. 
Then, an "indicator of effectiveness," must be something that 
measures whatever effectiveness is contingent upon — which, by 
definition, is a "criterion of effectiveness." In other words, 
indicators of effectiveness measure how an organization stands on 
criteria of effectiveness. 

At the same time, (1) a given measure may be used as an 
indicator for more than one criterion of effectiveness; and (2) a 
given criterion may serve more than one kind of effectiveness — 
logic and reason prevailing. 

The fifth premise is that the paradigm should be able to 
incorporate any and all criteria that have been employed as 
standards of organizational effectiveness. 

The Paradigm 

The paradigm is basei on six questions or issues that are 
essential to any assessment of effectiveness: 
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(1) Whose perspective are we concerned with? 

(2) What is the domain or criteria of assessment? 

(3) What is the referent against which effectiveness is to be 
judged? 

(4) What is the appropriate level and unit of analysis? 

(5) What should be the appropriate time frame? 

(6) What are the appropriate types and sources of data? 
Purely theoretical discussions of effectiveness generally 

raise the third question first. The logic being that the nature 
or characteristics of effectiveness should be one's first 
consideration. However r our ordering (at least with respect to 
the first three questions) reflects our belief that more often 
than not the administrator is responding to the demands of his 
constituencies rather than basking in their admiration for his 
judgments. Our ordering is intended to provide some additional 
insight regarding what "is" and "is not" being considered as 
critical by different constituencies. 

Whose perspective are we concerned with? 

Organizational effectiveness criteria will differ depending 
on whose (or which) viewpoint is taken. Clearly r what is most 
important to students is likely to differ from what is most 
important to their parents , faculty r tho community r the stater and 
so forth. 

For example r students are likely to be concerned about an 
institution's educational climate (e.g., its sense of communityr 
academic emphasis, vocational emphasis r social opportunities r 
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propriety, etc.), amenities, location, etc. Parents may be more 
concerned about cost, vocational emphasis, quality, religious 
emphasis, and so on. Faculty are likely to be more concerned 
about the institution's working environment (e.g., autonomy, 
conflict, morale, etc.), salaries, stature, etc. The State, at 
least with respect to public institutions, will be concerned about 
costs, efficiency, program duplication, etc. 

We turn to Cameron (1978, p. 11) for a formal statement of 
the problem: 

There is general agreement that effectiveness criteria always 
represent someone's values and biases, but there are 
conflicting opinions about who should determine effectiveness 
criteria and who should provide data for their measurement. 
Some investigators advocate relying on major decision makers 
and directors, or the organization's "dominant coalition," to 
generate the criteria and to supply effectiveness information 
(Gross, 1968; Pennings & Goodman, 1977; Price, 1968, 1972; 
Yuchtman and Seashore, 1967). Others have suggested that 
these top administrators or managers have narrow and biased 
perceptions and, therefore, that a broad range of 
constituencies should be tapped (Katz and Kahn, 1978; 
Pfiffner & Sherwood, 1960; Steers, 1975). Still another 
group (Bass, 1952; Priedlander & Pickle, 1968; Reinhard, 
1973: Scott, 1977) points out that constituencies outside 
the organization are relevant for criteria generation 
inasmuch as derived goals (Perrow, 1961), "macroquality" 
criteria (Reinhart, 1973), or information concerning the 
organization's contribution to the "supersystem" (Katz and 
Kahn, 1978) are obtained from that group. 

What this means is that the criteria of organizational 
effectiveness are likely to differ as a function of who you are 
talking to. So, you ask, which constituents should one listen to? 
The answer is that you have to listen to all of them. However, 
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it's extremely unlikely that ycu can satisfy all of them-- 
especially since their demands may contradict one another. 



What is the domain or criteria of assessment? 

Our research suggests that most if not all the criteria that 
have been discussed with respect to organizational effectiveness 
fall into one of four domains: goal achievement, managerial 
processes, organizational climate, and environmental adaptation. 
We believe that two of the most important outcomes of the current 
research effort are the (1) identification of most of the criteria 
that have been discussed with respect to organizational 
effectiveness, and (2) identification/development of domains that 
subsume and explain them. However, since dealing with the basic 
question ("What is the domain or criteria of assessment?") 
constitutes a large part of the current research effort, we shall 
address this problem in a separate section. 

What is the referen t against which effectiveness is to be judged? 

Organizational theory suggests that effectiveness criteria 
can be distinguished along three dimensions. One dimension is 
whether effectiveness criteria are normative or descriptive. A 
second dimension is whether criteria are static or dynamic. A 
third dimension is whether the criteria are universally valued. 

The normative/descriptive dimension distinguishes between 
whether a given criterion is prescribed or derived. Prescribed 
criteria denote characteristics or qualities that "experts" 
believe organizations must/should possess in order to be 
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effective. In the context of postsecondary education the most 
obvious example of normative or prescribed criteria are 
accreditation requirements. Derived criteria generally refer to 
goals set bj^ the organization itself, rather than foi the 
organization as in the case of prescribed criteria. However, the 
distinction between prescribed and derived criteria can become 
quite blurred since organizations can adopt prescribed criteria as 
goals to be valued in-and-of themselves. The interested reader 
will find more detailed discussions in Thompson (1967), and Steers 
(1977). 

The static dynamic dimension reflects one's biases regarding 
what constitutes organizational effectiveness. Employing static 
criteria reflects the belief that once an organization achieves 
the criteria, it is effective. Examples of static criteria 
include meeting institutional objectives for (a) student 
performance on standardized tests, (b) endowment campaigns, (c) 
faculty perceptions of campus morale, etc., but can also include 
accreditation. Static criteria are generally found under the Goal 
Achievement and Organizational Climate dimensions of this 
paradigm. 

Employing dynamic criteria reflects the belief that the 
organization is in a constant state of flux so that what's 
critical to effectiveness is the organization's ability to monitor 
and adapt to changing conditions. Examples of dynamic criteria 
include security from external threats, control over environment, 
bargaining position, flexibility, and adaptability. 
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The third dimension of the referent issue is whether a given 
criteria is held as universally valuable by everyona in an 
organization, or whether it is contingent on the values of a 
particular constituency. 

What is the appropriate unit of analysis? 

The unit of analysis is primarily determined by the criteria 
employed, which in turn is determined by the interests of key 
constituencies. For example, state legislators may be concerned 
with total institutional costs, program costs, or total costs p'=i 
full-time equivalent student. Institutional managers may be 
concerned with institutional productivity generally, and/or with 
the productivity of departments considered individually. 
Businesses may judge an institution's effectiveness in terms of 
the skills of graduates from particular departments. The general 
effectiveness or quality of Student Servir.es may be judged or 
effectiveness could be judged on the individual components— e.g. , 
financial aid, housing, food services, health, counseling, etc. 

We find a formal statement of the problem in Cameron (1978, 
p. 12): 

Bidwell and Kasarda (1975), Evan (1972), Hirsch (1975)r and 
Katz and Kahn (1978) are among those who advocate relying on 
the supersystem or the external "organizational set" to 
establish effectiveness criteria (effectiveness is defined as 
the ability of the organization to adapt to, manipulate, or 
fulfill expectations of the external environment); whereas 

B writers such as Scott (1977), Steer? (1977), Webb (1974), and 
Weick (1977) suggest that criteria should relate to the 
organization as a unit (effectiveness relates to the goals, 

■ processes, or characteristics of the organization itself). 

■ Pennings and Goodman (1977) propose an approach to 

I 
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effectiveness which focuses on organization subunits 
(organic-^ational effectiveness is associated with the 
contributions of and the coordination among subunits 
Argyris (1962), Kaufman (1960) , Lawler, Hall, and Oldham 
(1974) and others focus on individual performance as criteria 
of organizational effectiveness (organizational effectiveness 
is assumed to be indicated by individual behaviors and 
satisfaction) . 

What should be the appropriate time frame? 

The time frame component is concerned vith the fact that we 
must decide whether we are concerned with short, middle, or long- 
term conditions. Assessment may produce very different results 
when applied to each of these periods — even if we employ the same 
set of criteria. 

What are the appropriate types and sources of data? 

The appropriate types and sources of data are largely driven 
by responses to the five previous issues. However, within this 
issue we must deal with three specific but interrelated questions. 
The first is concerned with what type of data is collected. 
Examples include (a) questionnaire data — institution developed, 
published; (b) archival records? and (c) indicators — e.g., 
measures of efficiency, productivity, community impact, etc. The 
second component concerns whether assessment data is subjective 
(perceptual) versus objective (empirically verifiable). And, 
third, when assessment data is perception-based, who are 
appropriate respondents — for example, students (past/present, 
full-time/part-time, undergraduate/graduate, etc.), department 
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heads, faculty, trustees, heads of households, members of business 
and industry, transfer institutions, state and federal agencies, 
the military, etc. 
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Section 5 
Domains of Effectiveness 



The organi. -national effectiveness literature contains scores 
of effectiveness models—for example, the Rational Goal Model, 
Systems Resource Model, Cost/Benefit Analysis, Management By 
Objectives, Operatrons Research, Organizational Development, Open 
Systems, etc. The models generally assume that the ultimate 
criterion of effectiveness is the degree to which an organization 
realizes its goals, m the context of postsecondary education 
these goals generally refer to teaching, public service, and 
research. 

However, the supporters of all but the Goal Model argue that 
"goal achievement" is not a viable criterion since (1) the 
demands placed on organizations are generally so dynamic and 
complex that it is not possible to define a small number of goals 
that are meaningful; (2) organizations and subunits within 
organizations can pursue multiple goals that may be in conflict 
With one another; (3) the relative importance of goals, let alone 
the goals themselves may change over time; (4) there may be 
significant differences between official and operative goals; (5) 
goal achievement is extremely difficult to measure; and (6) goal 
attainment is only one of the requirements that an organization 
must meet to be viable— e.g., it must acquire resources, implement 
managerial structures, etc. 

The critics of the Goal Model argue that we must turn to 
other criteria i-^ assess crganisiitio.mi ef rectiveness . 
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Ironically, these other criteria become the "goals" of an 
effective organization. That is, they are viewed as the criteria 
of an effective organization in-and-of themselves as opposed to 
means of achieving the ends for which the organization was 
established. 

For example, the Systems Resource Model (Seashore and 
Yuchtman, 1967, p. 898) defines effectiveness in terms of an 
organization's aciiity "in either relative or absolute terms, to 
exploit its environment in the acquisition of scarce and valued 
resources." The Open Systems Model (Etzioni, 1964) defines 
effectiveness in terms of an organization's adaptability, 
efficiency, growth, resource acquistion, and external support. 
The Managerial Process Model (Koontz and O'Donnel, 1972) views 
effectiveness primarily in terms of how the organization's 
managers carry out their responsibilities with respect to 
staffing, controlling, planning, and directing. And, the 
Organizational Development Model (Campbell, 1974) suggests that 
organizational effectiveness is primarily a function of morale, 
cohesion, and consideration for human resources. 

A review of effectiveness models suggest that the same 
criteria are often employed by more than one model. In addition, 
the same set of data may be used to measure organizational status 
on different criteria. For example, the Organizational 
Development Model holds that organizational effectiveness is 
contingent on high morale. At the same time, however, data on 
employee morale could be used as an indicator of employee 
satisfaction with managerial evaluation standards— a criterion of 
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Managerial Process Models. Hence, in using a particular criterion 
and data we must be clear about what they are mtenoed to 
*neasure.l 

Rather than work with dozens of effectiveness models, we have 
chosen to identify the criteria employed by the most frequently 
discussed models and develop a framework that usefully 
incorporates them. A review of the literature suggested that most 
models (1) define effectiveness in terms of, and employ 
criteria that focus on--the extent to which an organization does 
one of the following: 

1) achieves its goals 

2) perforins certain mr.njgerial functions 

3) fosters a supportive climate 

4) adapts to its environment. 

This set of domains finds a good deal of support in the 
literature— particularly in the Competing Values Approach 
developed by Quinn and Rohrbaugh (1981, 1983). However, before 
describing the characteristic?, of the dimensions and criteria 
included in the framework several issues need to be discussed. 
The first concerns the relationship between the framework and the 
Competing Values Model (CVM). Quite simply, the dimensions of our 
framework follow directly from the CVM. 

J-tt would t>e easy to suggest that increasing morale might be 

taken as an organizational goal — whereupon, organizational 
effectiveness would be a function of the extent to which this goal 
was achieved. However, a careful reading of the 1 terature 
pertaining to the Rational Goal Model suggests that its focus is 
concerned with macro goals of the type found in charters and 
mission statements. This suggests that morale is more 
appropriately viewed as an effectiveness crj'-.erion of one of the 
other effectiveness models. 
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The value of the CVM is that it provides insight into both 
the nature of, and the relationships between, different 
effectiveness models and criteria. The CVM suggests that most 
effectiveness models and criteria can be differentiated on three 
factors: (1) flexibility versus control, (2) internal versus 
external focus, and (3) means versus ends. Rather than attempt to 
write our own summary of the CVM, we have taken the liberty ox 
including a sumnary (Appendix 1) written by Faerman and Quinn 
(1985). Knowledge of the CVM is not essential to using the 
measures and indicators reported in this document. However, 
knowledge of the CVM vtll provide the reader with useful insight 
about criteria and models of organizational effectiveness. 

The second issue is concerned with the generic nature of the 
criteria included in the Compendium of Measures (Appendix 3). The 
Compendium identifies indicatorb and measures that may be used to 
assess an organization's status on criteria of effectiveness. 
Criteria were assigned to domains as a function of the primary 
emphasis of the models they derive from. However, for purpo-es of 
parsimony, generic categories (and labels) were employed for 
criteria concerned with similar if not identical underlying 
behaviors or conditions. 

For example, the Compendium treats the criteria of morale and 
esprit as synonyms even though some researchers talk about 
organizational morale and others organizational esprit. Both are 
included as a single criterion under a more general criterion or 
construct that ^'j referred to as "consideration and support." 
Other criteria included under this criterion include intimacy. 
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aloofness, permeability, and warmth. Hence, t reader may not 
find the names of all the criteria associated with a particular 
effectiveness model in the Compendium. The behaviors underlying 
these criteria should, however, be represented by one of the 
generic criteria. 

The reader may recollect that the fif*-h premise of the 
paradigm is that it must be able to incorporate any and all 
criteria that have been taken as indicative of organisational 
effectiveness. Two conditions make this a challenge. The first 
follows from the previously discussed need to employ generic 
criteria. Researchers may use different language to describe 
criteria that are essentially the same as those in the Compendium. 
That is, they are concerned with the same behaviors. Thus, the 
Compenr'-um may appear to fail to include criteria deemed critical 
by a particular researcher when the problem is acttially semantic. 

The second condition follows from the fact that the Paradigm 
focuses on the assessment of "overall" institutional effectiveness 
rather than the effectiveness of particular institutional 
components (for example, the education department, student 
services, financial man/*gement, administration, etc). However, 
judgments oi effectiveness are often based on one or more specific 
institutional behaviors. The challenge results from the fact that 
the link between a specific focus and the generic criteria is not 
always obvious. 

However, the Compendium was specifically built to accommodate 
the fact that judgments of effectiveness are often based on the 
pefcrmance of organizations generally, as well as the performance 
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of individual and aggregated organizational components. 
Unfortunately, the reader may have to struggle with identifying 
the links since we have not yet identified how the focus of many 
specific assessments translate into the general criteria. 

The Compendium is constructed so that one can select a 
criterion from a particular effectiveness domain and automatically 
know (1) how effectiveness is defined (i.e., in terms of goal 
achievement, managerial processess, organizational climate, or 
environmental adaptation), and (2) why performance on the 
criterion is important (i.e., because it indicates the extent to 
which the organization is effective vis-a-vis the (a) 
achievement of its goals, or (b) fostering of a supportive 
environment, or (c) performance of certain managerial functions, 
or (d) adaptation to its environment. 

At the same time, however, one can employ a criterion tha'- is 
commonly treated as a standard of effectiveness in one domain as a 
standard in other domains as long as there is a direct, logical 
relationship between the criterion and the new domain. That is, 
as long as one can explain (1) why the criterion is an appropriate 
standard for the domain, and (2) identify measures of the 
criterion that are meaningfully related to the domain being 
considered. 

The discussion that follows is intended to serve the reader 
in several ways. First, it identifies most of the criteria 
commonly discussed as being indicative of organizational 
effectiveness and the kinds of effectiveness they are generally 
used to measure- -that is, goal achievement, managerial processes. 
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organizational climate, and environmental adaptation. Second, the 
Compendium reports indicators for most of the criteria that have 
been identified. Third, it may provide the reader with some 
useful insight regarding criteria and/or domains of effectiveness 
that have not been of concern to the reader, but that may be 
important to the health of your organization. 
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Goal Achievement 



The criteria which comprise the Goal Achievement domain are 
generally employed by models that view or define effectiveness in 
terms of the extent to which an organization "achieves its goals." 
Goals, in this context, refer to statemerts that describe the 
general purposes of the organization such as those likely to be 
found in an organization's charter, mission statement, and public 
statements made by the organization's leaders. Perrow (1961) 
refers to these as the "official" as opposed to the "operative- 
goals of an organization, in the context, of higher education 
th.ese would generally pertain to such things as the education of 
students (teaching), research, and public ser'-ice. 

Campbell (1974, p. 6) describes models which define 
effectiveness in terms of goal achievement as follows: 

The goal centered view make a reasonably explicit assumption 
that the organization is in the hands of a reasonably 
rational set of decision makers who have a set of goals in 
mind which they wish to pursue. Further, these goals are few 
enough in number to be manageable and can be defined well 
enough to be understood. Given that goals can be thus 
identified it should be possible to plan the best management 
strategies for attaining them. Within this orientation, the 
way to assess organizational effectiveness vould be to 
develop criterion measures to assess how well the goals are 
being achieved. Thero are a nunber of variations of the goal 
centered view. The management by objectives tradition (e.g., 
Odiorne, 1965, 1969) ea it is usually practiced tends to fall 
within this category. The recently renewed movement toward 
cost/benefit analysis (Rivlin, 1971) is an ambitious attempt 
to assess the actual utility of specific goals. 
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Cunningham (1977, p. 465) describes the Rational Goal Model 
as follows: 

The rational goal approach focuses on the organization's 
ability to achieve its goals. Evaluation criteria are 
derived from a definition of the qo&ls an organization is 
expected to achieve (Etzioni, 1960, 1964; Ghorpade, 1971; 
Perrow, 1961, 1967, 1968; Simon, 1964; Thompson and McEwen, 
1958; Warner, 1967). These criteria are determined by 
various factors (Price, 1968; Thompson, 1958). One common 
practice is to use the formal statement of goals found in 
charters, manuals, and other documents. Informal, but 
operative goals constitute other useful criteria. Still 
others may be derived from conceptualization of societal 
missions or functions of the organization... 

An organization's goals are identified by establishing the 
general goal, discovering means or objectives for its 
accomplishment, and defi..ing a set of activities for each 
objective. The organization is evaluated by comparing the 
activities accomplished with those planned for... 

It is extremely important that we establish what is meant by 

"goals" within the current context. Hence, even at the risk of 

losing the reader's interest, we feel obliged to call on Parsons 

(1956, p. 63) for further clarification: 

...An organization is defined as a social system oriented to 
the attainment of a relatively specific type of goal, which 
contributes to a major function of a more comprehensive 
system, usually the society. Such an organization is defined 
as an institutionalized value system, above all defining and 
legitimizing its goal, and of the mechanisms by which it is 
articulated with the rest of society in which it operates. 

...An organization is a system which, as the attainment of 
its goal, "produces" an identifiable something which can be 
utilized in some way by another system; that is, the output 
of the organization is, for some other system, an input... 
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As previously noted, the Goal model has numerous critics. 
These criticisms aside, however, it seems reasonable to believe 
that judgments regarding the effectiveness of many organizations, 
particularly publicly supported organizations like colleges and 
universities, are often based on the extent to which they achieve 
their goals. For current purposes it seems reasonable to assume 
that the goals of colleges and universities are teaching, 
research, and public service. However, goals stated this 
generally don't get us very far. 

One way out of this problem is tc assume that we may assess 
such globally stated goals vis-a-vis the achievement of finer 
grained goals or, the objectives for achieving them.l This 
assumption buys us a great deal. Researchers have developed more 
than 80 different inventories of goals, objectives, and outcomes 
appropriate for postsecondary education (Lenning, Lee, Micek, 
Service, 1977). 

We have chosen to deal with the problem by adopting an 
inventory that seems at least as good, if not better, than most. 
The inventory was developed at NCHEMS by Lenning, Lee, Micek, and 
Service (1977). One major benefit of adopting this inventory is 
that it provides examples of measures or indicators for each of 



^Webster (1981, p. 785) defines an objective as "something 

toward which effort is directed: an aim or end of action: GOAL. 
According to the Encyclopedia of Educational Evaluation (Anderson, 
Ball, and Murphy, 1975, p. 179), "goals are concerned with 
ultimate outcomes and are usually phrased in general or global 
terms. Objectives are narrower and usually short-ranged; they are 
statements of student behaviors that, taken together, are thought 
to contribute to the envisioned final goals." 
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the elements included in the inventory. A description of the 
NCHEMS outcome structure is included in Appendix 2. 

The NCHEMS model is stated in terms of "outcomes." However, 
these can easily be adapted to our needs by assuming that outcomes 
are simply objectives (or criteria) that have been achieved. We 
can turn any outcome into an objective simply by treating it as 
something to be achieved. For example, the first outcome in the 
NCHEMS structure is "Economic Access—The amount of openness or 
ease of admittance to economic opportunities, advancement." 
Stated as an objective (or criteria of effectiveness), it would 
read, "openness or ease of admittance to economic opportunities, 
advancemert . " 

The NCHEMS structure identifies five major categories of 
outcomes: (1) Economic, (2) Human Characteristics, (3) Knowledge, 
Technology, ar.J Art Forms, (4) Resource and Service Provision, and 
(5) Other Maintenance and Change. These categories are described 
in Table 2. The outcomes included under each of these categories 
are described in Table 3. 

The outcomes in Tables 2 and 3 are numbered in a scheme that 
differs slightly from the scheme used by Lenning and his 
associates. The first digit references the effectiveness domain— 
(1) Goal Achievement, (2) Managerial Processes, (3) 
Organizational Climate, (4) Environmental Adaptation. The second 
digit references major categories of objectives in the domain. 
The third digit references subcategories within the major 
category. The fourth digit references a specific type of 
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The Major 


Table 2 

"Type-of-Outcome" Catgory Names and Definitions 


1 


Category 
Code Number 




[ 


1100 


Economic Outcomes — Maintenance or change in 
economic characteristics and conditions of 
individuals, groups, organizations, ard 
communities, e.g.. In economic access, in economic 
mobility and independence, in economic security, 
and in income and standard of living. 




1200 


Human Characteristic Outcomes — Maintenance or 
change in human makeup and characteristics (other 
than knowledge and understanding) of individuals, 
groups, organizations, and communities, e.g., 
aspirations, competence and skills, affective 
characteristics, perceptual characteristics, 
physical and physiological characteristics, 
personality and personal coping characteristics, 
recognition and certification, and social roles. 


I 


1300 


Knowledge, Technology, and Art Form Outcomes — 
Maintenance or change in the knowledge and 
understanding, technology, or the art forms and 
works possessed or mastered by individuals, groups, 
organizations, and communities, e.g., discoveries 
and inventions, technical developments, syntheses 
and reformulations of knowledge, new schools of 
thought in art and works created in those new 
traditions, renovation of art works. 


1 


1400 


Resource and Service Provision Outcomes — 
Maintenance or change in the direct resources and 
services (other than those included above) provided 
to individuals, groups, organizations, and 
communities, e.g., providing facilities, events, 
advisory assistance, analytic assistance, teaching, 
health care, and leadership. 




1500 


Other Maintenance and Change Outcomes — Examples 
would be: maintenance or change in the format, 
arrangement, activity, or administrative operation 
of an organization or Institution; maintenance or 
change in the aesthetic/cultural level of the local 
community; maintenance or change in family or 
community activities, practices, and traditions. 
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Table 3 

Goal Achievement Outcomes-Criteria (1000) 



1100 ECONOMIC OUTCOMES 



1110 Economic Access and Independence Outcomes — Outcomes that 
relate to the entrance into, obtainability , flexibility, 
and levels and amounts of monetary or pecuniary 
situations, conditions, and characteristics, 

1111 Economic Access — The amount of openness or ease of 
admittance to economic opportunities, advancement, 

1112 Economic Flexibility, Adaptability, and Security — 
The amounts of self-sufficiency, liberty, frugality, 
thrift, self-government , confidence, certainty, 
safeguards, stability, and adjustment that are 
exhibited in economic matters, 

1113 Income and Standard of Living — Amount of profits, 
return on investment, necessities and comforts of 
life, wealth, and other signs of economic "well- 
being" that are obtained or possessed. Included is 
direct support provided to individuals and the 
community through local purchases by the educational 
institution and through staff salaries and wages. 

1120 Economic Resources and Costs — Outcomes that relate to the 
amount and type of material, energy, effort, people, 
organization, and other economic assets that are available 
or that are expended in economic activities and 
production. 




1121 Economic Costs and Efficiency — The amounts of 
sacrifice, effort, expenditure, and waste present in 
economic activit.^es and production. 

1122 Economic Resources (including employees) — The assets 
available that can aid economic production, 
distribution, and gain. 

1130 Economic Production — Outcomes that relate to the creation 
of goods, services, and economic value. 

1131 Economic Productivity and Production — The value of 
goods and services that are created or produced by 
and within specific enterprises of "audiences" or 
clients of the educational institution, and 
especially in relation to the resources expended in 
the enterprise. 

1132 Economic Services Provided — Amount and type of 
direct-assistance activities provided by the 
educational institution or its subunits in the 
economic area. 

1140 Other Economic Outcomes — An example would be that a 

company with a large payroll located in this community 
rather than another similar community because there is a 
more prestigious college here. 
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1200 HUMAN CHARACTERISTIC OUTCOMES 

1210 Aspirations -Levels, patterns, and directions (in persons, 
groups, organizations, or communities) of interests, 
desires, drives, ambitions, goals, and intentions. 



1220 



1211 



Desires, Aims, or Goals— Pl;»r^c . conditions, things, 
or other ends that individuals and/or groups crave, 
toward which they have ambition, or that they inteAd 
to reach because of importance to them. 

1212 Dislikes. LikeSf an<^ Interests — The persons or types 
Of persons, objects, content areas, occupations and 



1213 



other things and situations for which there is a 
preference or antipathy. 

Motivation or Drive Level— ThP intensity of strivinq 
toward a goal that is elicited by a need or other 
stimulus. 



1214 Other Aspirational Outcomes 

Competence and Skills -Levels, patterns, and direction of 
ability, capability, proficiency, and talent of different 
Kincis • 

1221 Academic Skills— The amount of ability or competence 
in taking tests, earning good grades, persisting in 
college, etc., without regard to the amount of 
cognitive learning that has taken place. 

^222 Citizens hip and Family Membership Skills — ThP 

ability or competence to perform relative to the 
rights, duties, and privileges of a member of a 
family, community, state or nation; for ea^ample, 
competence in managing family finances, being an 
effective consumer, and evaluating political 
issues. 

1223 Creativity Skills— The amount of ability or 
competence in designing, producing, or otherwise 
bringing into existence original perspectives, 
explanations, and implementations. 

1224 Expressi on and Communication Skills — The amount of 
ability or competence in conveying information, 
attitudes, or emotions on a one-to-one basis and/or 
to large or small groups or populations, by whatever 
media, in order to inform, challenge, uplift, and/or 
persuade, etc., and in receiving and interpreting 
such communications— through reading, writing, 
speaking, listening, touching, body movement, 
silence, and cultural arts like acting, painting, 
sculpturing, singing, playing musical instruments. 
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1200 HUMAN CHARACTERISTIC OUTCOMES (convinued) 

1225 Intellectual Skills — The amount of ability or 
competence in formulating and analyzing problems, 
comprehending and understanding, synthesizing 
information, evaluating information, implementing a 
solution to a problem, and in locating, retaining, 
and filtering relevant knowledge. 

1226 Interper sonal, Leadership, and Organizational 
Skills — The amount of ahn-i^-y ;^«%n»p^»^nrc in 
effectively living and interacting with others, 
social organizing, being a congenial friend and 
companion, establishing courses of action for 
others, and influencing others to follow. 

1227 Occupational Skills — The amount of ability or 
competence in the special, unique skills required by 
particular occupations, and in seeking, gaining, and 
maintaining a particular level and kind of 
employment . 

1228 Physical and Motor Skills — The ability or competence 
in tasks requiring physical coordination, dexterity, 
manipulation, and other muscular or motor skills. 

1229 Other Skill Outcomes — Examples are the ability to 
teach effectively, to handle one's leisure, etc. 

1230 Morale, Satisfaction, and Affective Characteristics — 
Levels, patterns, and directions of characteristics 
typified by emotion. 

1231 Attitudes — The disposition or tendency to respond 
either positively or negatively to particular 
persons or types of persons, things, situations, 
etc. It is a predisposition to act in a certain 
way. 

AND 

Values — A strong preference based on a conception of 
what is desirable, important, and worthy of esteem. 
Values affect an individual's actions and thoughts 
toward others. 

1232 Beliefs, Commitments, and Philosophy of Life — The 
acceptance and internalization of particular 
propositions or declarations; the particular things 
that one is convinced are true. The held view of 
whfct "man" is, the purposes and reasons for a 
person's existence, and the system of principles and 
laws that should govern his/her thought, morals, 
character, and conduct or behavior. Included is the 
promotion of and the adherence to the conventions, 
practices, and teachings of religious organizations 
or sects. ' 
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1200 HUMAN CHARACTERISTIC OUTCOMES (continued) 



3 Feelings and Emotions — The disposition or tendency 
to respond or not respond subjectively to stimuli 
and the ability to control or not control such 
expressions, i.e., feelings of anguish or distress, 
anticipation, anxiety, concern, contentment, 
empathy, excitement, fear, frustration, happiness 
and joy, humor, lethargy, love, pleasure, 
satisfaction, sorrow, etc. 

* Mores, Cu stoms, and Standards of Conduct — Social and 
cultural practices, rules, and conventions designed 
to guide personal and corporate behavior. They have 
strong ethical or moral significance according to 
tradition and are enforced by social disapproval of 
violations. 



l'*35 Other Affective Outcomes 

40 Perceptual Characteristics — Level s . patterns, and 

directions of consciousness, awareness, and sensitivity 
exhibited, and the view(s) or concept(s) of self, others, 
surroundings, events, ideas, etc. 

1241 Perceptual Awareness and Sensitivity — The amount of 
consciousness or awareness of, or sensitivity to, 
stimuli that are exhibited by individuals or 
groups . 



■ 1242 Perceptio n of Self — The view held about oneself; the 

■ characteristics that are perceived, i.e., self 

■ concept. 

I 1243 Perceptio ns of Others — The manner in which other 

■ individuals and particular groups of others are 

viewed or perceived; the characteristics that are 

■ perceived. 

1244 Perceptio ns of Things — The view one holds (i.e., the 
characteristics noted) of ideas or other things 

■ being examined with the physical senses. 

1245 Other Perceptual Outcomes 

I 1250 Personal ity amJl Personal Coping Characteristics — Levels . 

patterns, and directions of human conditions, factors, and 
^ traits related specifically to the mind and mental 

■ processes (other than skills, knowledge, and 

■ understanding). 

■ 1251 Adventuro usness and Initiative — Willingnegg to take 

■ chances and risks; how daring an individual is; 

willingness to take a stand or speak out; 

m willingness and capacity to initiate personal action 

H or to become a.;tively involved. 
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1200 HUMAN CHARACTERISTIC OUTCOMES (continued) 
1252 

exhibited. 



Autonomy and Independei^ce --The amount of freedom 
from control and influerce of others that is 



^253 Dependabi lity and Responsibility — The amount of 

reliability, trustworthiness, and accountability for 
own behavior that is exhibited. 

•••254 Dogmatism, Author itarianism, and Qpen-Mindedness — 
The amount of open-mindedness, assert iveness , 
una^sertiveness, and/or unquestioning obedience to 
authority that is exhibited. 

1255 Flexibil ity and Adaptability — The amount of 
adjustment to new and changing situations and 
circumstances that is exhibited. 

1256 Habj. .s— The tendency uo perform certain actions or 
to behave in characteristic, automatic ways. 

1257 Psychological Functioning — The amount of 
psychological adjustment, contact with reality, 
self-understanding, and self-actualization (optimum 
self-realization) that is exhibited. 

1258 Tolerance and Persistence — The amount of endurance, 
tenacity, forbearance, patience, and restraint that 
is exhibited. 

1259 Other Personality and Personal Coping Outcomes 

1260 Physical and Physiological Characteristics — Levels . 
patterns, and directions of human body traits and 
processes (other than skill functioning). 

1261 Physicg l Fitness and Traits — Physical and 
physiological characteristics such as toughness, 
endurance, strength, speed, flexibility or 
dexterity, physical energy, muscular control, size, 
vocal characteristics, etc. 

1262 Physiological Health — The physical well-being of 
individuals; how well tne system of normal bodily 
operations is functioning. 

1263 Other Physical or Physiological Outcomes 

1270 Status, Recoanitionp ^n d Certification — Lpv^T s . patterns, 
and direction, concerning recognition of accomplishments, 
power, prestige, reputation, etc. 

1271 Completi on or Achievement Award — A certificate, 

diploma, or some other award for having completed a 
course or program, for some demonstrated 
pr<,riciency, or for accomplishment of some type. 
•"O -50- 
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1200 HUMAN CHARACTERISTIC OUTCOMES (continued) 



Credit Recognition — Formal or informal 
acknowledgement of work completed or of confidence, 
trust, approval, etc. 

Image, Reputation ^ or Status — The amount of fame, 
distinction, respect, and standing in the eyes of 
the profession, the community, or some other group. 

Licensin g and Certification — Formal written 
authority that a person or t^rm is qualified and has 
met the test to practice some skill or speciality 
occupation. 

1275 Obtainin g ri Job or Admission to a Follow-Up 
Program — Success in being selected for a 
postgraduate employment position or a special 
education at a higher level. 

1276 Power and/or Authority— The amount of acknowledged 
authorization or ability to influence, command, 
enforce obedience, or set policy as a right of rank, 
position, delegated jurisdiction, skill, strength, 
wealth, etc. ^ 

1277 Job, School, or Li fe Success — Evidence of success in 
one s occupation or career, in graduate or 
professional school, or in some other aspect of 
one's life that is covered in any of the above 
categories . 

^-^^ Other St atus, Recognition, and Certification 
Outcomes 



1273 
1274 



'''' Social Activities and Roles — Levels . patterns, and 

directions of social functions assumed and carried out. 

1281 Adjustment to Retiremen t— Alter 4 ng self anc 
lifestyle to meet the needs and adapt to the 
limitations of the retirement years. 

1282 Affiliations — Finding appropriate organizations and 
institutions to join and associate with, and being 
accented by them. 

1283 Avocational and Social Activities and Roles — 
Finding, pursuing, an^^ achieving rewarding nonwork 
activities, hobbies, and parts to play in society, 
and exhibiting that pattern of behavior that is 
expected of persons having the status that has been 
earned. 



1284 Career and Vocational Activities and Roles — 

Exhibiting the patterns of behavior expected and/or 
that are needed for th*» part in the "world of work" 
that has been accepted or entered into. 
O -51- 
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1200 HUMAN CHARACTERISTIC OUTCOMES (continued) 

1285 Citizens hip Activities and Roles — Facilitating and 
contributing to governmental functions and to the 
overall well-being of individuals, the community! 
and larger society. 

1286 _Fam ily Activities .^nd Roles — Contributing to and 
facilitating family functions, i.e., parent roles, 
sibling roles, son/daughter roles, etc. 

1287 Friendsh ips &nd Relationships — Socially interacting 
with and entering into and sustaining intimate, in- 
depth, and satisfying associations with others. 

1288 Other Activity and Role Outcomes 
1290 Other Human Characteristic Outcomes 

1300 KNOWLEDGE, TECHNOLOGY, AND ART FORM OUTCOMES 

131? General Knowledge ard Understanding — Familiarity with, 
analysis and comprehension of, and application of facts 
and principles across broad areas of study — breadth of 
knowledge and understanding— as a result of dissemination 
through educational teaching/learning activities. 

1311 Knowledge and Understanding of General Facts and 
Terminology — Knowing about and understanding, and 
having an adequate vocabulary to be able to 
describe, the reality, existence, and circumstances 
of particular sensory (observed, heard, felt, etc.) 
phenomena, objects, people, products, events, 
conditions, etc., or components thereof. 

1312 Knowledge and Understanding of General Processes — 
Knowing about and understanding customs, rules and 
standards for judgments, guidelines, processes, 
methods, procedures, techniques, trends, and other 
ways of applying and making use of terminology and 
facts. 

1313 Knowledge and Understanding of General Theory — 
Knowing about and understanding princ-ipies and 
generalizations, theoretical formulations, 
hypotheses, suppositicn, conjecture, etc. 

1314 Other General Knowledge and Understanding 

1320 Specialized Knowledg e and Understanding — Familiarity with, 
analysis and comprehension of, and application of facts 
and principles in particular specialized fields of study- 
depth of knowledge and understanding— as a result of 
dissemination through educational teaching/learning 
activities. _52„ 



1300 KNOWLEDGE, TECHNOLOGY, AND ART FORM OUTCOMES (continued) 



1321 Knowledg e and Understanding of Specialized Facts and 

Terminology — Knowing ahmit anH nnri^re»anrij r^g =.r,^ 

having an adequate vocabulary to be able to describe 
the reality, existence, and circumstances of 
particular sensory (observed, heard, felt, etc.) 
phenomena, objects, people, products, events, 
conditions, etc., or components thereof. 

1322 Knowledge and Understanding of Specialized 
Processes— Knowing about and understanding customs, 
rules and standards for judgments, guidelines, 
processes, methods, procedures, techniques, trends, 
and other ways of applying and making use of 
terminology and facts. 

1323 Knowledge and Understanding of Specialized Theory — 
Knowing about and understanding principles and 
generalizations, theoretical formulations, 
hypotheses, supposition, conjecture, etc. 

1324 Other Specialized Knowledge and Understanding 

1330 Research and Scholarship — KnowleHt^P and understanding, 

techniques, and physical products resulting from basic and 
applied research and scholarship. 

1331 Research and Scholarship Knowledge and 
Understanding — The discovery, development, 
preservation, and professional dissemination of 
knowledge and understanding resulting from 
activities conducted in basic and applied research 
and scholarship. 

1332 Research and Scholarship Products -~Applied 
techniques (for example, a new therapy treatment in 
the field of medicine or a new technique in the 
field of music) and physical products (for example, 
a new or refined serum) developed from basic and/or 
applied research and scholarship. 

1340 Art Forms and Works — Reproducing and preserving existing 
artistic forms and works, and developing new or revised 
artistic forms and works. 

1341 Architecture — Outcomes involving the design for 
construction of buildings, landscape, living 
complexes, etc. 

134 2 Da net* — Outcomes involving preservation or 

development of forms, works, and performances in the 
art of dance. 



r <^ 
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1300 KNOWLEDGE, TECHNOLOGY, AND ART FORM OUTCOMES (continued) 



1343 Debate and Oratory — Outcomes involving preservation 
or development of forms and performances in the 
oratory arts. 

1344 Drama — Outcomes involving the preservation or 
development of forms, works, and performances in the 
professional and amateur theatrical arts. 

1345 Literatu re and Writing — Outcomes involving the 
preservation or development of forms and works in 
the production of prose, verse, and other writings. 

1346 Music — Outcomes involving the preservation or 
development of forms, works, and performances in the 
professional and amateur theatrical arts. 

1347 Painting , Drawing, and Photography — Outcomes 
involving the preservation or development of forms 
or works in the graphic and pictorial arts. 

1348 Sculpture — Outcomes involving the preservation or 
development of forms or works in the carving, 
chiseling, casting, modeling, or other sculpturing 
areas . 

1349 Other Fine Arts 

1350 Other Knowledge. Technology, and Art Form Outcome s 



1400 RESOURCE AND SERVICE PROVISION OUTCOMES 

1410 Provision of Facilities and Events — The availability, use, 
and participation in campus happenings, buildings, 
equipment, and other resources by students, other 
individuals, and particular groups or communities. 

1411 Provision of Facilities — Availabi li ty and use of 
campus grounds, buildings, rooms, equipment, etc. 

1412 Provisio n or Sponsorship of Events — Availability and 
participation in happenings on the campus or off 
that are provided or stimulated by the college or 
one of its components. 



1420 



erJc 



Provision of Direct Services — The availability, use, and 
receipt by students, other individuals, and particular 
groups or communities of assistance, care, or other 
service. 

1421 Teaching--Activities and prog- ^ms designed to 

instruct and to impart knowledge, skills, attitudes, 
etc. 
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1400 RESOURCE AND SERVICE PROVISION OUTCOMES (continued) 

^^22 Advisory and Analytic Assistance — Activi i-i and 

programs designed for the purpose of (upon request) 
ottering suggestions, recommendations, counsel, 
information, calculations, and studies. 

1423 Treatment, Care, and Referral Services — Helping and 
direct assistance services, other than those above, 
provided by the institution, institutional units, 
and/or institutional staff. 

1424 Provision of Other Servires — An example would be 
direct civic leadership provided to the community. 
Another example would be offering keypunching 
service. =» ^r- a 

1430 0^^^^ f ^"^ Service Provision Outcon.P« -An example 
would be tne attention and good will the college draws to 
tne local community because it is located there. 

1500 OTHER MAINTENANCE AND CHANGE OUTCOMES 

Aesthetic-Cultural Conditions— Proc^.„,- or bringing 
about changes in tastes, level and kinds of aesthetic- 
cultural emphasis, aesthetic-cultural availability and 
opportunities, aesthetic-cultural activity and 
participation, etc. 

1520 Organizational Format, Activity, and Operations — For 

orga- izaticns, groups, and systems (and their components), 
maintenance >r change in organizational communications, 
operational nethods and interaction, operational 
effectiveness, organizational relationships, 
organizational arrangement and configuration, 
organizational activities and programs, and other such 
organizational characteristic outcomes. 

1530 Other Maintenance and Changre— nni-r^orr.oc not covered by any 
of the other subcategories of "Maintenance" and "Change" 
in this dimension of the Outcomes Structure. An example 
might be "destruction of life support in the 
environment." 
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criterion within the major and cub-major category. This coding 
scheme is used in all discussions and tables that follow. 

Examples of measures that can be used to assess achievement 
of these outcomes/objectives/criteria are reported in the 
Compendium of Measures (Appendix 3). The numbers reported under 
each of the criteria refer to indicators that are described in 
Appendix 4. The source of each indicator is reported in Appendix 
5— along with all the criteria an indicator may be used to 
measure. We urge the reader to check these sources for complete 
descriptors of their utility, strengths and weaknesses. 

For example, the data in Appendix 3 indicate that Economic 
Access (category llli) can be measured by the percentage of 
students obtaining their first full-time job in the field of their 
choice within a specified time after graduation (indicator #1); 
and, the number of alternatives for an entry level job open to 
minority group graduates compared to minority group nongraduates 
(indicator #2). Both of these indicators were taken from the 
NCHEMS Outcome Structure. 

Appendix 6 includes a description of two online databases 
that should be extremely useful to the reader. Both are 
maintained by the Bibliographic Retrieval Service (BRS). The 
f_rst includes the most recent version of all tests evaluated for 
the Buros Mental Measurements Yearbooks, its label is "MMYD." 
The MMYD database contains information on more than 1100 tests. 

The second database was developed by the Educational Testing 
Service. It contains data on more than 5000 instruments. Its BRS 
label is "ETSF." The reader may access these databases to 
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identify instruments chat may be used tc assess criteria discussed 
throughout the Compendium. 

For our purposes, the thorniest component of the Outcomes 
Structure is category 1520— Other Maintenance and Change Outcomes: 
Organization Format, Activity, and Operations. The category 
suggests that the effectiveness of a postsecondary institution can 
be judged as a function of the extent to which it achieves its 
goals regarding "maintenance or change in organizational 
communications, operational methods and interaction, operational 
effectiveness, organizational relationships, organizational 
arrangements and configuration, organizational activities and 
programs, and other such organizational characteristic outcomes." 

We have said that goal-centered effectiveness models view or 
define effectiveness in terms of the extent to which organizations 
achieve their goals. However, treating "maintenance or change in 
organizational conditions" (for example, budgeting, planning, 
flexibility, etc.) as organizational goals (1) is not consistent 
with the type of goals that are supposed to be included in this 
domain— teaching, researrh, and public service; (2) allows the 
domain to swallow-up the criteria included in all the other 
domains; and, perhaps of greatest importance (3) obscures the 
reasons for being concerned about these other conditions or 
criteria in the first place. 

For example, taking the implementation of a particular 
budgetary process as an "organizational goal" obscures the fact 
that budgeting is part of a larger managerial process, the concern 
of which is the viability ot the entire organization and the 
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achievement of its goals. Hence, measures and indicators for 
assessing organizational conditions will not be included in thi 
section. The reader who feels the necessity of treating 
organizational conditions as organizational goals will find 
measures for assessing them in the following sections. 
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Managerial Processes 



A second class of models adopts the classical management 
approach to organizational effectiveness. That is, effectiveness 
is viewed or defined in terms of the extent an organization's 
managers perform certain managerial functions which are deemed 
critical to the achievement of organizational objectives. The 
models which form this domain (Bureaucratic Model, Internal 
Processes, Classical Management Principles) are all normative in 
nature — they assume that there are certain activities that all 
organizations must do to achieve their objectives. 

The critical underlying assumption of Managerial Process 
Models is that organizations are rational entities, in which the 
effective mobilization and direction of internal processes and 
activities naturally leads to the achievement of organizational 
goals and objectives. Therefore, effectiveness criteria and 
measures for this class of models focus on the internal processes 
and mechanisms that produce results, rather than the achievement 
of outcomes. Further, this view of organizational effectiveness 
suggests that internal processes (for exampl«=-, planning) can be 
objectively defined, rationally implemented, and subsequently 
measured. Effectiveness is then determined by the congruence 
between intended (clearly defined and structured) processes and 
realized results. 

The models which form this domain generally adopt Weber's 
(1947) assumption that individuals are rational, willing actors in 
organizations that are defined as the sum of directed individual 
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efforts. Individuals are viewed (Reeser, 1973, p. 1) as "the 
instruments through which managers utilize capital, materials, 
plant, and equipment." The presumed instrumentality of 
individuals underscores and defines the role of managers as those 
who are charged with concentrating and controlling the efforts of 
ethers . 

The criteria for harnessing individual efforts into a single, 
concerted organizational endeavor are planning, organizing, 
directing, controlling, and staffing. These criteria and their 
measurement are also based on rational principles of 
organizational behavior. This suggests that a shared 
characteristic of all these criteria is that in order to become 
effective, all internal processes are expected to (1) be clearly 
and precisely stated, (2) define the responsibilities of all 
members of the enterprise, and (3) formally state the 
relationships between individual organizational members. 

The Criteria 

Planning, in Managerial Process Models, is the mechanism by 
which managers attempt to influence future activities by actions 
taken in the present (Hutchinson, 1971). The major components 
(criteria) of planning include the identification of 
organizational objectives, and the selection of policies, 
procedures, and methods designed to meet those objectives. 
Policies are defined as general principles for organizational 
activities. Procedures are the chronological sequence of steps to 
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be followed in order to meet an objective. Methods refer to a 
particular step of a procedure that is to be performed. 

Organizing is the mechanism that develops the set of formal 
relationships between and among organizational members. 
Organizing requires the development and formal articulation of 
managerial span of control, channels of communication, lines of 
authority, managerial and employee responsibilites, division of 
labor, and centralization or decentralization of tasks. 

Directing is largely a communication process whereby leaders 
learn what motivates members of the organization, and use that 
information to direct activities. The components of directing 
include the (1) communication of clear, concise expectations for 
successful task completion, as well as a statement of the 
standards for rewards contingent on successful task completion; 
(2) articulation of employee development opportunities; and (3) 
identification and implementation of techniques and methods for 
motivating staff. 

Control is the process of feedback that allows organizational 
leaders to evaluate the effect of past efforts, estimate the 
effect of present efforts on future outcomes, and estimate the 
effects of corrective actions taken. The components of control 
include establishing (1) standards by which achievement of 
organi:;ational activities (for example, planning, organizing) can 
be evaluated in the present and in the future; (2) mechanisms by 
which information gathered needed for evaluation can be gathered; 
and (3) channels of distribution for feedback reports. 
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Staffing is the process whereby the organization's personnel 
needs are identified and met. The major components of staffing 
include job definition, recruitment, and assessment. The purpose 
of the staffing function is to ensure that organization roles are 
filled by personnel able and willing to occupy them. 

The criteria shown in Table 4 are supplemented by principles 
of good management suggested by Koontz and O'Donnel (1972). We 
have used these principles as a means of gaining a little more 
insight and specificity regarding the criteria for achieving 
effectiveness from a managerial process perspective. For exampl 
Koontz and O'Donnel describe the "Authority Level Principle" as 
follows (p.. 411): Maintenance of intended delegation requires 
that decisions within the authority competence of an individual 
manager be made by him and not be referred upward in the 
organization structure. The principle is restated as a criterion 
(#2215) in Table 4 as, "Decisions within the authority competence 
of an individual manager are made Ly him/her and not referred 
upward in the organization structure." 

Unfortunately, we have not completed our research with 
respect to all the subcriteria pertinent to Managerial Process 
Models. Hence, the Compendium shows indicators only for the major 
criteria shown in Table 4. 
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Table 4 

MANAGERIAL PROCESSES (2000) 

2100 Planning — the mechanisms by which managers attempt to 

influence future activities by actions taken in 
the present 



2110 



Primacy of Planning Principle — planning 
other managerial functions . 



preceeds all 



2120 Planning Process Principle— individuals charged with 
planning understand and utilize consistent planning 
premises. 

2130 Timing Principle— plans are structured to provide an 

appropriately timed, intermeshed network of derivative 
and supporting programs. 

2131 Time Line — plans include time lines. 
2140 Objectives 

2141 Organizational — clear, written statements of 
organizational mission that delineate domains of 
activity and outline responsibilities of, and 
expectations for, providers and consumers of the 
educational product. 

2142 Department & Subunit — clear, written 
statements ot department and subunit objectives 
that are consistent with the overall goals and 
mission of the organization. 

2143 Policies, Procedures, and Methods — policies, 
procedures, and meethods are designed to meet 
organizational objectives. 

2200 Organizincy— the mechanism that devslops the set of formal 

relationships between and among organizational 
members 

2210 Span of Control Principle — the span of managerial 
control is clearly articulated . 

2211 Scalar Principle — there is a clear line of 
authority trom the top manager in an enterprise 
to every subordinate position. 

2212 Delegation Principle — the authority delegated to 
an individual manager is adequate to assure his 
ability to accomplish the results expected of 
him. 
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2213 Absolute Responsibility Principle — the 
responsiblity of the subordinate to his superior 
for performance is absolute, and no superior can 
escape responsibility for the organisation 
activities of his subordinate. 

2214 Parity of Authority Principle — the responsibility 
for actions is not greater than the authority 
delegated. 

22.15 Authority-Level Principle— H^mci r.ne within the 

authority competence of an individual manager are 
made by him and net referred upward in the 
organization structure. 

2220 Empxoyoe R esponsibilities — th*» responsiblities of 
managerial employees in all departments, units and 
subunits in the organization are clearly delineated. 

2230 Channels o f Con....unication — channpT s of communication 
are clearly articulated. 

2240 Division of Work— depan-mpni- . unit, and subunit 
responsibilities are clearly delineated. 

2300 Directing— the process whereby leaders learn what motivates 

members of the organization, and use that 
information to direct activities 

2310 Leadership Principle— managf^^e understand what 
motivates GUi'>ordinates . 

2311 Motivation — managers motivate employees. 

2320 Communication— expectai- inns for successful task 

completion are clearly communicated to employees. 

2330 Development—employee development opportunitjps are 
Clearly art Iculated. 

2400 Control— the process that allows leaders to evaluate the 
effect of past efforts, estimate the effect of 
present efforts on future outcomes, and estimate 
the effects of corrective actions takon 

2410 Standards — clear delineation of standards for 
evaluating the achievement of organizational 
activities 



2411 Evaluation — clear delineation of criteria 
which objectives ara evaluated. 



on 
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2412 Information Collection — mechanisms exist for 

collecting information pertinent to assessment. 

2420 Reflection of Plans Principle —controls are designed to 
deal with and reflect the nature and structure of 
plans. 

2430 Suitabili ty Principle —controls ire designed to reflect 
the place in the organization structure where 
responsiblity for action lies. 

2440 Distribution — channels are established for 

distributing the results of feedback reports, studies, 
etc. 

2450 Corrective Action— correctivp action is taken if-and- 
when objectives are not met. 

Staffing— the mechanism whereby the organization's personnel 
needs are identified and met 

2510 Recruitment — formal standards and procedures exist for 
selecting and promoting personnel. 

2520 ^Valuation — formal standards and procedures exist for 
assessing all positions in the organization. 
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Organizational Climate 



A third class of models view effectiveness primarily in terms 
of the extent to which organizations foster a supportive climate 
with respect to (a) meeting the needs of individual members, and 
(b) working together as a team. The models which employ this 
focus are generally associated with research in the field of 
Organizational Development (OD). The "Organizational Climate" 
domain reflects a slightly broader perspective then most of the 
Models associated with OD perspectives in that it also encompasses 
aspects of an institution's educational climate. 

Taking the broader concept of "climate" for the moment, 
Campbell (1974, p. 149) defines it as a "molar concept reflecting 
the content and strengt! of the prevalent values, norms, 
attitudes, behaviors, and feelings of members of a social system. 
Climate is thus a result of the transaction between individual 
members, with their idiosyncratic needs, abilities, goals, and the 
organizational structure." Li twin and Stringer (1968, p. 188) 
£rgue that understanding and manipulating an organization's 
climate is important because it "creates certain kinds of beliefs 
(expectancies) about what kinds of consequences will follow from 
various actions, and it indicates the kinds of satisfactions or 
frustrations (incentive values) that are available in a given 
situation . " 

Traditional OD is concerned with understanding and 
manipulating organizational climate vis-a-vis the use of 
intervention techniques that provide an organization's members 
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with a means of examining their behavior in the "here-and-now" 
(Bennis, 1969) — for example, T-groups, process consultation, 
confrontation, etc. However, our interest in OD models follows 
from their definitions of what constitutes organizational 
effectiveness, rather than in the intervention techniques 
themselves. 

OD theory generally recognizes the achievement of 
organizational goals as the ultimate criterion of organizational 
effectiveness. At the same time, however, OD theorists argue that 
in view of the previously noted shortcomings of taking goal 
achievement as "the" criterion of effectiveness, and because of 
the dynamic nature of organizations and their environments, the 
most effective organization is one whose human resources and 
energy are optimally m obilized to "achieve the organization's 
mission and, at the saire time, maintain a viable, growing 
organization of people whose personal needs for self -worth, 
growth, and satisfaction are significantly met at work (Beckhard, 
1969) ." 

Margulies and Raia (1972) point out that Organizational 

Development is essentially a systems approach to the total set of 

functional and interpersonal role relationships in 

organizations. They explain this as follows: 

...the organization system consists of three major elements 
cr subsystems: (1) the technical or task system, which 
includes the flow of work, the technology involved, the 
required task roles, and a number of other technological 
variables; (2) the managerial, or administrative system, 
which includes the organization structure, policies, 
procedures, and rules, the reward and punishment system, the 
ways in which decisions get made, and a large number of other 
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elements designed to facilitate the management process; and 
finally (3) the human, or personal-cultural system, which is 
primarily concerned with organizational culture, values, 
norms, and the satisfaction of personal needs. Also included 
in the human system are the informal organization, the 
motivational level of members, and their individual 
attitudes. It is the interaction of these three systems that 
produces the behavior and role relationships that affect 
organizational output. 1 

In a seminal review of the OD literature, Campbell (1974) 

identifies three assumptions that appear ^o underlie OD theory: 

...First, rapid and accelerating change is depicted as a fact 
with which both men and org?nizations must accept and cope 
(Bennis, 1969). The world is seen as changing in a variety 
of ways at an increasing rate. This implies that old ways of 
managing and organizing are no longer functional. The 
functional bureaucracy, in particular- is noted as being an 
outmoded model of organizations. Second, an optimistic point 
of view is taken toward the nature of man. Man is seen as 
reaching for growth, seeking self -actualization , and 
certainly much less in need of supervision in organizations 
than the conventional wisdom would imply. Almost any article 
or book dealing with OD will make this statement (Beckhard, 
1969; Bennis, 1969; French and Bell, 1973; Margulies and 
Raia, 1972), but McGregor's statement on Theory X and Theory 
Y (1960) is stili the most quoted version. Third, though 
this is less often made explicit, organizations are viewed as 
existing primarily, if not solely, for the benefit of the 
individual members of the organization. French (1972) 
states, "One value to which many behavioral scientist-change 
agents tend to give high priority, is that the needs and 
aspirations of human beings are the reasons for organized 
effort in society (p. 35)." 

Campbell suggests that in addition to these three general 
values or ass?amptions most OD researchers and practitioners 
Operate with some more specific set of organizational 

^Margulies and Raia, 1972, p. ■ 
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characteristics that define a healthy system. Several alternative 

lists of such characteristics are reported in Table ?. 

Reviewing the assumptions and normative goals of OD theory 

Campbell (1974, p. 19) concludes that from the OD perspective, the 

"effective organization:" 

...will be a'jare of, open to, and reactive to change. It 
will be searching for new forms and methods of organizing. 
It will have an optimistic view of its members, allowing them 
room to self-actualize and trusting them with the 
responsibility for their own efforts. It will also seek to 
insure the satisfaction of its members since that is its 
reason for existence. To these ends, conflict will be 
confronted, not avoided, and communication will occur freely 
and ef f ectivt^iy . 

In other words, OD models assume that if an organization can 
achieve the state characterized by a list such as Beckhard's, it 
will be effective as an organization and will be optimally 
equipped to carry out its mission. The criteria shown in Table 5 
are just a sample of those discussed in the literature. The 
biggest drawback to providing a unique list of such criteria is 
that different researchers use different labels for criteria that 
are concerned with very similar behaviors. Failure to address 
this problem would provide a list of criteria that would be 
several times longer than that found in Table 5. 

We have dealt with this problem by identifying eight major 
categories of criteria. The categories were developed by doing a 
"mental factor analysis" of the criteria found in OD models and 
instruments used for OD purposes. The categories and the criteria 
that comprise them are described in Table 6. When subcategories 




Table 5 

ALTERNATIVE SETS OF ORGANIZATIONAL EFFECTIVENESS 
CRITERIA OBTAINED FROM LITERATURE ON ORGANIZATION DEVELOPMENT 
from Campbell, Bownas , Peterson, and Dunnette (1974) 





ueptrioeijc. V9riaDl6 


Source 


1. 


High trust and support among 
organizational members. 


French (1972) 


2. 


Confrontation (not avoidance) 

yJL. pLOt/lcmb* 


N 


3. 


Knowledge based on authority as 
well as assigned role. 


tf 


4. 


Open communications 


n 


5. 


High satisfaction and enthusiasm 
for organizational members. 


N 


6. 


Frequent synergistic solutions. 


n 


7. 


Pr6s@nr^^ m^our^ i^Acn^nc ^ i 

v^Luup responsxDX 1 X ty 

for planning and implementation. 




1. 


OpeDr problem solving climate. 


Bennis (1969) pp. 36-37 


2. 


Role authority supplemented with 
authority based on competence. 


II 


3. 


Decision-making responsibility is 
located close to information sources. 


H 


4. 


High trust among persons and groups 
throughout organization. 


n 


5. 


Competition is relevant to work goals " 
and collaborative efforts are maximized. 


6. 


The reward system recognizes both 
achievement of organizational goals 
(profits or services) and development 
of people. 


It 


7. 


High sense of ownership of organi- 
zational objectives throughout work 
force . 


N 
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Managers manage according to relevant 
objectives rather than according to 
past practices* 



1. Communication of information is Schein (1965) 
reliable and valid. 

2. Internal flexibility and creativity " 
in accordance with information, 

3. High integration and commitment 
to goals of organization. 

4. Internal climate is characterized 
by support and freedom from threat. 



"The total organization, the Beckhard, 1969 

significant subparts, and individuals 
manage their work against goals and 
plans for achievement of these goals." 

"Form follows function (the problem, " 
or task, or project determines how the 
human resources are organized." 

"Decisions are made by and near the " 
sources of information regardless of 
where these sources are located on the 
organization chart." 

"The reward system is such that 
managers and supervisors are rewarded 
(and punished) comparably for: 
— short-term profit or production 

performance, 
— growth and development of their 

subordinates, 
— creating a viable working group." 

"Communication laterally and vertically " 
is relatively undistorted. People are 
generally open and confronting. They 
share all the relevant facts including 
feelings. " 
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••There is a minimum amount of 
inappropriate win/ lose activities 
between individuals and groups. 
Constant effort exists at all levels 
to treat conflict and conflict situations 
as problems subject to problem solving 
methods . •• 

"There is high "conflict" (clash of 
ideas) about tasks and projects, and 
relatively little energy spent in 
clashing over interpersonal difficulties 
because they have been generally 
worked through." 

"The organization and its parts see 
themselves as interacting with each 
other and with a larger environment. 
The organization is an "open system." 

"There is a shared value, and 
management strategy to support it, 
of trying to help each person (or 
unit) in the organization maintain 
his (or its) integrity and uniqueness 
in an interdependent environment." 

"The organization and its member 
operate in an action-research way. 
General practice is to build in 
feedback mechanisms so that 
individuals and groups can learn 
from their own experience." 
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Table 6 

Organizational Climate (3000) 



3100 Autonomy — degree to which the group is independent of 

other groups; self-determination of group activity (GDDQ) 

3110 Control — degree of group regulation of member 
behavior (GDDQ) 

3120 Flexibility — extent to which group activities are 
free from constraint by custom, tradition, written 
rules, or unwritten codes (GDDQ) 

3130 Responsibility — employee discretion in work, 
without supervisor chec)cing up (L&S) 

3140 Task Structure — the degree to which the methods 
used to accomplish tasks are spelled out by the 
organization (W&H) 

3200 Structure — emphasis on constraints, rules, regulations 
and formal procedures; orderliness, routine, 
formalization, standardization (L&S) 

3210 Production Emphasis — close, directive supervision 
(OCDQ) 

3220 Stratification — differentiation of internal status 
hierarchy (GDDQ) 

3230 Recognition and Feedback — degree to which an 

individual knows what his supervisor and management 
think of his work and the degree to which they 
support him (W&H) 

3300 Consideration and Support — perceived helpfulness and 

backing received from superiors, peers, and subordinates 



3310 Esprit — morale; social and achievement need 
satisfaction (OCDQ) 

3320 Intimacy — friendly social relations among members 
(OCDQ); closeness of acquaintanceship; familiarity 
with personal details of each other's lives (GDDQ) 

3330 Aloofness — high emotional distance from lead, j 
(OCDQ) 

3340 Permeability ~ openness of group to new members 
(GDDQ) 
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3350 Human Resources Primacy — concern for welfare and 
happiness of workers (ISR) 

3360 Warmth — friendliness within the work group and 
organization (L&S) 

3370 Potency -- centrality of group membership in the 
lives of members (GDDQ) 

3380 Status and Moral — the general feeling among 

individuals that the organization is a good place in 
which to work (W&H) 

3400 Synergy — (viscidity) cohesiveness ; absence of dissension 
and conflict; degree to which all members function as a 
unit (GDDQ) , cooperation 

3410 Hindrance — performance hindered by petty 
administrative details (OCDQ) 

3420 Participation — proportion of time spent in group 
activities (GDDQ) 

3430 Goal Consensus — degree to which group goal is 
unitary, and explicit to all members 

3440 Standards — perceived importance of organizational 
goals and performance standards (L&S) 

3450 Conflict — emphasis on working through rather than 
avoiding conflicts (L&S) 

3460 Identity ~ feeling of belonging to the organization 
and work group (L&S) 

3470 Achievement Fmphasis — the desire on the part of 

the organization to do a good job and contribute to 
the performance. 

35°° Reward Or ientation^ — reward/performance relationship — 
reflects the degree to which the granting of additional 
rewards such as promotions and salary increases are based 
on performance and merit rather than other consi-jerations 
such as seniority, favoritism, etc. (W&H) 

3510 Hedonic Tone — amount of pleasure afforded by 
membership (tddQ) 

3520 Motivational Conditions — presence and nature of 
organizational factors eliciting effort (ISR) 

3600 Oppenness vs. Def ensiveness — degree to which people try 
to cover tneir mistakes and look good rather than 
communicate freely and cooperate (W&H) 
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3610 Communications Flow — freedom of flow of task 
relevant information within and between grouos 
(ISR) 

3620 Security vs. Risk — reflects the degree to which 
pressures in the or^Janization lead to feelings of 
insecurity and risk (W&H) 

3630 Support for creativity, experimentation 

3700 Participation vs. Decision Centralization 

3710 Disengagement — group merely going through the 
motions; low involvemenL (OCDQ) 

3720 Decision Making Practices — decision making 

characterized by delegation and participation vs. 
centralization (ISR) 

3730 Lower Level Influence ~ amount of influence 

possessed by workers and first level supervisors 
(ISR) 

3740 Decision Centralization ~ the extent to which 
decision making is reserved for top management 
(W&H) 

3800 Educational C limate ; Emphasis on student learning, 
socilization, and motivation. 

3810 Student Learning . 

3811 Intellectual Climate — devotion to 
scholarship in humanities, arts, and social 
sciences (CCI) 

3812 Academic Climata — emphasis on academic 
excellence in humanities and physical sciences 
(CCI) 



3813 Academic Achievement — press for high student 
achievement (CCI) 

3814 Academic Organizatio n — emphasis on 
organization and structure in the environment 

3815 Vocational Climate — press for practical and 
conservative activities (CCI) 

Scholarship — perceived environmental press 
for academic achievement; selectivity of the 
institution; importance of getting acceptable 
grades (CUES) 
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3820 Socilization and Motivation. 

3821 Aspiration Level — expectation that students 
will set high goals (CCD 

3822 Self-Expression — opportunity to develop 
leadership ability and self-assurance (CCI) 

3823 Group Life — incidence of mutually supportive 
group activities (CCI) 

3824 Social Form — press for "proper" social 
behavior (CCI) 

3825 Student Dignity — degree of student autonomy 
and self-determination (CCI) 

3826 Party Climate ~ party atmosphere (CCI) 

3827 Awareness — perceived press for self- 
expression; artistic orientation; intellectual 
press (CUES) 

3828 Community — perceived precs for social 
activities; affiliation with faculty and other 
students (CUES) 



3829 



Propriety — press for social conformity; 
constraint; deference to tradition (CUES) 
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were taken from a particular instrument, the initials of the 
instrument are shown in parentheses. The instruments are 
identified at the ♦ind of this section. 

Only the major categories are referenced in the Compendium. 
This was done because (1) many of the subcategories are concerned 
with very similar behaviors; and (2) we have not had sufficient 
time to develop an adequate set of indicators fcr each of the 
subcategories. The reader requiring measures of a particular 
subcategory may turn to the (1) instruments noted as containing 
them; (2) ETS and MMYD databases described in Appendix 5. 

The last (eighth) major cat*»gory is Educational Climate. The 
subcategories shown under this category do not exhaust the list of 
criteria that are relevant of an institution's educational 
climate. However, thoy are a reasonably good sample. 
Unfortunately, we have not yet identified adequate indicators for 
each of the subcategories. The Compendium shows only a single 
indicator for Educational Climate. The indicator references 
standardized instruments such as those associated with each of the 
subcategories. Again, the reader requiring a measure of a 
particular subcategory may turn to the instrument from which it 
was initially taken (shown in parentheses); and/or the ETS and 
MMYD databases. 
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Instr'ixnents 

CCI — College Chavacteristics Index (Stern, 1970) 

CUES— College and University Environment Scales (Pace, 1968) 

GDDQ—Group Dimensions Descriptive Questionnaire (Hemphill and 
Westie, 1950) 

ISR — Survey of Organizations (LiJcert, 1967) 

T.,^S— Climate Questionnaire (Litwen and Strenger, 1968) 

OCDQ— Organizational Climate Descriptor questionnaire (Halpin and 
Croft, 1962) 

W&H — Campbell, Dunnette, Lawler, amd Weick (1970) 
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Environmental Adapation 



The last set of models to be considered view or define 
effectiveness in terms of the extent to which an organization 
adapts to its environment. The models which employ this focus 
(for example, Systems Resource ^Seashore and Vuchtman, 1967), 
Structural Functional (Perrow, 1967), Multiple Constituencies 
(Pennings and Goodman, 1977) all argue and assume that the only 
appropriate way to unders' and assess organizational 
effectiveness is by adopting an open systems perspective. 

We believe that one of the simp? est yet informative 
descriptions of open systems models is found in Hall (1972, p. 
23). He suggests that the distinction between the closed and 
open-systems approaches to organizations has its roots in 
Gouldner (1959, p. 405), who distinguished between the "rational" 
and "natural system:" 

The natural-system model regards the organization as a 
"natural-whole," or system. The realization of the goals of 
the system as a whole is but one of several important needs 
to which the orgfnization is oriented. 

Hall goes on to explain fiat one of these "important" needs 
is survival which can le?.d to neglect or distortion of goal- 
seeking behavior. He writes (p. 23): 

...The organization is seen as emergent, with organizational 
goals playing a relatively minor role in the directions in 
which the organization emerges. The natural systems approach 
stresces the interdependence of the parts of organizations, 
noting that even a planned change in one part will have 
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important, and usually unanticipated, ramifications for the 
rest of the system. 

The existence of multiple effectiveness models based on a 
syt ems view meano that each must differ at least slightly from 
one another in terms of their definitions cf effectiveness and the 
criteria they employ. However, common to all models are the 
assumptions that in order for an organization to survive it must 
(1) do more than achieve its goals~e.g., acquire resources, 
implement managerial structures, (2) acquire resources and 
allocate them efficiently, and (3) manage its subsystems so they 
are in harmony and coordinated to v^ark together. 

Seashore and Yuchtman developed one of the first systems- 
based effectiveness models. After citing all the previously noted 
criticisms of goal centered (that is, closed system) models of 
effectiveness they proposed the following: 

IWe] define the effectiveness of an organization in terms of 
its bargaining position, as reflected in the ability of the 
organisation, in either absolute or relative terms to exploit 
its environment in the acquisition of scarce and valued 
resources. 

The concept of "bargaining position" implies the exclusion of 
any specific goal (or function) as the ultimate criterion of 
organizational effectiveness. Instead it points to the more 
general capability of the organization as a resource-getting 
system. 1 

Seashore and Yuchtman argue, in accord with systems theory, 
that "input of resources" is only one of the three major cyclic 

^Seashore and Yuchtman, 1967, p. 898 ~ 
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phases in the system of organizational behavior. The other two 

being throughput and output. However, they contend that the 

bargaining position of an organization is a function of, and 

reflects, all three phases of organizational behavior. 

A view which emphasizes the role of organizational 

subsystems and constituencies in determining effectiveness was 

developed by Pennings and Goodman (1977). They write: 

. . .Organizations are seen as open systems having excha-^.ge 
relationships with their environment and with subsystems that 
render a contribution to the whole and to each 
other. . .Organizations are also seen as comprising interest 
groups, or constituencies, which make claims on the 
organization. . .Organizational subsystems, in this open 
systems view, both determine organizational effectiveness, 
and play a role as constituency in defining its criteria. 

...Organizational effectiveness is associated with the 
contributions of subunits .. .Organizational effectiveness is 
likely to be a function of the extent to which the subunits 
meet their task requirements as well as the extent to which 
their activities are coordinated. . .If each subunit were 
independent, organizational effectiveness would equal the 
combined effectiveness of all subsystems... 

The Organizational Environment 

Before we can identify appropriate criteria and indicators of 
how well an organization has adapted to its environment, we must 
first decide what is meant by the "organization's environment." 
Hall (1972) describes an organization's environment as everything 
"outside of" a particular organization— "Climactic and gc graphic 
conditions, other organizations, the state of the economy... "2 
He distinguishes between an organization's "specific" and 



^Hall, 1972, p. 9. 
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"general" environment. The specific environment is composev.^ of 
the organizations and individuals with which an organization is in 
direct interaction. The general environment is viewed as being 
comprised of eight different, but overlapping conditions: 

1) technological conditions 

2) legal conditions 

3) political conditions 

4) economic conditions 

5) demographic conditions 

6) ecological conditiors 

7) cultural conditions 

8) social conditions 

At the other end of t.ie spectrum, Starbuck (1976) and Child 
(1972) argue that organizations cannot be distinguished from their 
environments. That, since organizations largely invent or enact 
their environment, they cannot be distinguished from it. In 
Starbuck 's words: 

Assuming organizations can be distinguished from their 
environments distorts reality by compressing into one 
dichotomy a melange of continuously varying 

phenomena. . .Organizations • environments are largely invented 
by organizations themselves. Organizations select their 
environments from ranges of alternatives, then they 
subjectively perceive the environments they inhabit. 3 

Child (1972) argues: 

The environment of an organization cannot be satisfactorily 
defined without reference to what Levine and White have 
called "organizational domain." This consists of specific 
goals which organization decision makers wish to pursue and 
the functions which they cause an organization to undertake 
in order to implement these goals... 



^Starbuck, 1976, p. 1069 
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...We have argued that the analysis of organization and 
environment must recognize the exercise of choice by 
organizational decisionmakers. They may well have power to 
enact their organization's environment (Weick, 1969, p. 63). 
Thus, to an important extent, their decisions as to where the 
organizations operations shall be located, the clientele it 
shall serve, or tne types of employees it shall recruit, 
determine the limits of its environment . ^ 

We generally side wivh the position taken by Starbuck and 
Child. However, the potential focii of assessment are more easily 
identified by assuming that an organization's environment is 
defined by the conditions described by Hall. Irrespective of 
which position is taken, either 's interpretation of what 
constitutes an organization's environment means that the focus of 
assessment for a given criterion will probably differ as a 
function of which environmental condition we are concerned about. 

For example, determining organizational efficiency relative 
to existing technological conditions entails different questions 
and indicators than would be appropriate for assessing efficion'jy 
relative to existing economic, social, or cultural conditions. 
Unfortunately, we have not yet (1) determined how the focus 
changes for all of the criteria included in this domain, or (2) 
identified indicators for assessing them. However, the Compendium 
does identify measures that can be used to arsess eome of the 
criteria relative to specific environmental conditions. 
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Criteria of Adaptation 

The effectiveness models which form the basis of this domain 
generally view or define effectiveness in terms of (1) how well an 
organization has adapted to its environment— that is, its 
"adaptation;" and (2) its ability to adapt to changing or 
potential environments—that is, it«= "adaptrbility. " Resource 
acquisition and utilization is the primary focus in both 
conditions. However, in the first condition (adaptation), the 
concern is with resource acquisition and utilization "in-and-of 
Itself." In the second condition, the focus is on resource 
acquisition and utilization for purposes of generating 
organizational "slack." 

According to the models which form the basis for this domain, 
in order for an organization to be effective it must be efficient 
and productive; it must have the ability to acquire resources, and 
actually acquire them; it must satisfy its clients; it must be 
able to search out, correctly interpret, and respond to the real 
properties of the environment; it must have some control over, and 
enjoy some security from, threats to its well-being. These 
criteria form the basis for the Environmental Adaptation Domain 
(Table 7). Procedures and measures for assessing them are 
reported in the Compendium. 

While none of the criteria shown in Table 7 are easily 
assessed, two are particularly troublesome: efficiency and 
productivity. Efficiency is generally thought of in terms of a 
ratio that reflects some aspect of unit performance to the costs 
incurred for that perforir.ance. Economists, however, talk about at 
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Table 7 

ENVIRONME'^TAL ADAPTATION (4000) 



4100 Efficiency— generally thought of in terms of a ratio 

that reflects some aspect of unit performance to the costs 
incurred for that performance. 

4200 Productivity— is usually defined (a) as the quantity of or 
volume o: the major product or service that the 
organization provides; (b) value of outputs relative to thp 
value of inputs. 

4300 Bargaining Position --abilitv of the organization to 

exploit it^ environment in the acquisition of scarce and 
valued resources. 

4310 Tangible Resources — number . quality, and type of 
resouces: 

4311 Physical Plant & Eouipment --e.g. . buildings, 
grounds, book value, replacement cost, etc. 

4312 Support Services— e.g. . counseling, finanical .aid, 
housing, food, computer, library, etc. 

4313 facilities— buildings (e.g., gross area, assignable 
area, age, replacement cost, condition, etc.), 
rooms (e.g., classrooms, laboratories, offices, 
special use, general use, etc.). 

4314 Faculty and Staf f — ehtnicitv. sex, appointment 
status, type of appointment, type of position, 
terure status, educational credentials, etc. 

4315 Students — e.g. . full-tirr.e, part-time, 

level, adult, race, age distribution, geographic 
origin, educational credientials , objectives, 
aptitude, etc. 

4316 Financial— assets (e.g., ca..h, investments, 
accounts receivable, inventories, etc.), 
liabilities (6.g., accounts payable, deferred 
revenues/nredite, owner equity, etc.) 

4317 Collections— library resources or library holdings 
(e.g., physical units, audiovisual materials, 
distribution, acquisitions, etc.) 

4318 Educational Programs — regular . continuing 
education, quality, accreditation, etc. 
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4320 Intangible Resources — e.g., goodwill, community 

involvement and support, faculty and staff morale. 

4400 Client Sa tisfaction — ability of the organization to meet the 
needs of Its various constituencies. 

4500 Capacity T o Test Reali ty— ability of the organization's 

decisionmakers to perceive and correctly interpret the real 
properties of the environment 

4600 Control Over The Environment — ability of the organization's 
decisionmakers to enact its environment. 

4700 Flexibility/Readiness — ability of the organization to change 
in response to environmental changes. 



r o 



-86- 

91 



least three different aspects or interpretations of efficiency: 
1) technical efficiency, which refers to how resources are used in 
the production process; 2) pric^ (or allocative) efficiency refers 
to how resources are allocated given their prices; and 3) 
preference (or value) efficiency, which refers to the relationship 
between the amount of resources used and the mix of outputs. 
These three aspects of efficiency taken together can be referred 
to as production efficiency. This is to be distinguished from 
exchange efficiency, which refers to the delivery of products and 
services to customers. 

To complicate th'.ngs further, a search of the ERIC database 
provides more than 100 entries for the combined descriptors 
"efficency" and "higher education." The articles dealt with 
topics that included: enrollment systems, multi-campus systems, 
fund-raising, cost, learning, automation, scheduling, management, 
financial aid, library practices, campus planning, energy 
conservation, pricing, etc. 

In view of the complexity of the issues surrounding the 
efficiency criterion, even for experts in the field, we feel 
obliged to point out that our understanding and treatment of the 
problem is simplistic. Indicators for the general criterion of 
efficiency (4100) are shown in the Compendium. However, we have 
. c attempted to identify measurer for either general (production, 
exchange) or specific subcriteria (technical, price, preference). 
The interested reader will find an excellent discussion and 
summary of the efficiency problem in higher education in Lindsay 
(1982). 
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In technical terms, "productivity" according to Wallhaus 
(1975), is defined as the value of outputs relative to the value 
of inputs. However, he goes on to explain that: 



...this definition is interpreted in a variety of ways as it 
relates to different decisions, policy issues, and persons 
involved in higher education. These different perceptions of 
productivity are understandable considering the wide range of 
products— student growth and development, community services, 
new knowledge and art forms (Micek and Wallhaus, 1973; Gross 
and Grambsch, 1968)~and the variety of possible roles and 
missions of higher education institutions: as instruments of 
social change, environments for human relations, manpower 
factories, sorting and screening devices, and educational 
utilities, for example (Weathersby, 1971; Peterson, 1973). 
Thus, to understand productivity in higher education one must 
place the technical definition in the context of the 
decisions being made and the policy issues surrounding the 
enterprise. 

Utilizing an input/process/output model of higher education, 
this definition can be examined in light of the following 
questions: What is the mix of inputs? who pays for these 
inputs? What is the lidx of outputs? Who benefits from these 
outputs? And what is the relationship between inputs ai i 
outputs in terms of the technologies employed?^ 

To illustrate the problem, a search of the ERIC database 
produced more than 400 entries for productivity and higher 
education. The articles on productivity dealt with topics that 
include: faculty workload, long-range planning, library planning, 
statewide planning, adult education, staff utilization, enrollment 
trends, self esteem, research, etc. Again, W3 must apologize to 
the reader for our lack of expertise. WJiile we have included a 



"^^Ihaus, 1975, p. 1. 
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handfull of productivity measures in the Compendium, our treatment 
of the subject is simplistic at best. 

A third criterion which may benefit from further explanation 
is "Bargaining Power," or the ability of the organization tc 
exploit its environment in the acquisition of scarce and valued 
resources. This criterion is the key component of the Systems 
Resource Model (Seashore and Yuchtman, IS??). The criterion is 
fundamentally concerned with "the more general capability of the 
organization as a resource-getting system. "2 Measurement of 
this criterion requires two different types of data. One, focuses 
on the tangible resources the organization has actually acquired 
or has access to — for example ^ personnel, facilities, technology, 
money, etc. The second is concerned with intangible resources 
that the organization has earned and/or can call upon. These 
Would include such things as goodwill and community support. This 
second category of resources is largely a function of what an 
organization has done for its constituencies, and what it can do 
in the future. 

Before concluding this chapter we would like to suggest one 
additional environmental condition net discussed by Hall— academic 
conditions. Inclusion of this category in conjunction with the 
"Resources" criterion forms the basis for several of tho criteria 
previously discussed as components of effective colleges and 
universities — particularly accreditation requirements. 
Recognizing that accreditation agencies generally view these as 
indicators of an organization's ability to achieve its goals, they 



really are criteria that an organization must respond to in order 
to meet its academic obligations. 
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Appendix 1 

The Competing Values Approach 

From "Effectivenss: The perspective from organizational theory, "by 
S.R. Faerman and R.E. Quinn, 1985. 

In a radical departure from previous analytic studies 
designed to derive dimensions of organizational effectiveness, 
Quinn and Rohrbaugh (1981, 1983) posed the question, "How do 
individual theorists actually think about the construct of 
effectiveness?" Thus, they focused on the cognitive structure of 
organizational theorists and researchers, rather than oi. specific 
organizational structures or processes. 

In a series of studies, Quinn and Rohrbaugh had 
organizational theorists and researchers make judgments regarding 
the similarity or dissimilarity between pairs of effectiveness 
criteria. Starting with a comprehensive list of effectiveness 
criteria compiled by Campbell (1977), Quinn and Rohrbaugh 
organized and reduced this list by asking panelists to identify 
any criterion that was: (1) not at the organizational level of 
analysis, (2) not a singular index, but a composite of several 
criteria, (3) not a construct, but a particular 
operationalization, or (4) not a criterion of organizational 
performance. Of the original 30 criteria, 13 were eliminated. 
Panelists were then asked to compare each of the 17 criteria to 
each of the other 16, and to make judgments regarding the 
similarity or dissimilarity between each pair. The data were 
analyzed data using a multidimensional scaling algorithm. Results 
of the analyses suggested that organizational theorists and 
researchers share an implicit theoretical framework, or cognitive 
map, and that the criteria of organizational effectiveness can be 
sorted according to three axes, or competing value dimensions (see 
Figure 1) . 

As can be seen in Figure 1, there are two primary axes around 
which the criteria of performance are organized. The vertical 
dimension suggests that people differentiate criteria according to 
the degree of flexibility or control that is implied. Hence, at 
the top of the diagram the emphasis is on decentralization and 
differentiation. T..e effectiveness criteria we find at this end 
of the continuum are value of human resources, adaptability and 
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readiness. At the bottom of the diagram the emphasis is on 
centralization and integration. Effectiveness criteria at this 
end of the continuum are concepts like stability, control, 
planning and goal setting. The horizontal axis ranges from a 
concern with external criteria related to the competitive position 
of the overall system, to internal criteria related to the 
maintenance of the socio-technical system. Hence on the right are 
such criteria as productivity and efficiency, external support, 
and resource acquisition and growth. On the left are cohesion, 
morale, information management and communication. 

While Figure 1 is a two dimensional graphic, the data suggest 
that the criteria of effectiveness have a three dimensional 
organization. The third dimension is related to the difference 
between short term processes and long term outcomes. Thus 
planning and goal setting are viewed as a means to productivity 
and efficiency; adaptability and readiness are viewed as a means 
to growth, resource acquisition and external support; information 
management and communication are viewed as a means to stability 
and control; cohesion and morale are viewed as a means to 
increasing the value of human resources. 

The scheme is called the competing values framework because 
the criteria seem to carry a conflictual message. We want our 
organizations to be adaptable and flexible, but we also want them 
to be stable and controlled. We want growth, resource 
acquisition, and external support, but we also want tight 
information management and communication. We want an emphasis on 
human resources, but we also want an emphasis on planning and goal 
setting. While none of these pairs is mutually exclusive in an 
empirical sense, they tend to reflect different orientations or 
perspectives in a cognitive sense. 

Because the competing values framework consists of bipolar 
relationships, it has a clearly defined doi.,-in. These 
relationships consist of juxtaposed dimensions of effectiveness 
criteria which act as conceptual oppositions. They are not 
opposites in a mutually exclusive sense (i.e., short-tall, 
fat-thin). They are concepts which are cognitively distant from 
one another. A curious aspect of this specification of competing 
values is that it makes clear both the differences and the 
similarities in organizational thinking; it provides a 
simultaneous integration and differentiation. This clarification 
of competing values can be seen in the three basic axes, discussed 
above, and also in the four models represented in the four 
quadrants. in examining the criteria associated with each of the 
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four quadrants, it becomes clear that the criteria of 
effectiveness graphically define the development of organizational 
theory. As discussed above, different periods of organizational 
theory have highlighted different aspects of organizational 
functioning, while various attempts at integrating che literature 
(Scott, 1977; Seashore, 1979; Cameron, 1979) have identified these 
periods, they have not shown the relationships between models. 
The competing values framework makes clear these relationships. 

The human relations model places a gr<.at deal of emphasis on 
flexibility and internal focus, and stresses such criteria as 
those in the upper left quadrant: conesion and morale (as meant) 
and human resource development (as ends). The open systems model 
places a great deal of emphasis on flexibility and external focus, 
and stresses such criteria as those in the upper right quadrant: 
flexibility and readiness (as means) and growth, resource 
acquisition and external support (as ends). The rational goal 
model places a great deal of emphasis on control and external 
focus, and stresses such criteria as those in the lower right 
quadrant: planning and goal setting (as means) and productivity 
and efficiency (as ends). The internal process model is 
represented in the lower left c^uadrant. it places a great deal of 
emphasis on control and internal focus, and stresses the role of 
information management and communication (as means) and stability 
and control (as ends). 

Because each model is embedded in a particular set of 
competing values, each has a polar opposite with contrasting 
emphasis. The human relations model with effectiveness criteria 
reflecting flexibility and internal focus stands in stark contrast 
to the rational goal model's value-based stress on control and 
external focus. The open systems nodel, based on flexibility and 
external focus, runs counter to the internal process model, which 
reflects a focus on control and internal focus. Parallels among 
the models are also important. The human relations and open 
systems models share an emphasis on flexibility. The open system 
and rational goal models are primarily concerned with an external 
focus. The rational goal and internal process models are rooted 
in the value of control. Finally, the internal process and human 
relations models share an internal focus. 
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Appendix 2 

A Structure for the Outcomes of Postsecondary Education 
from Oscar T. Lenning, Yong S. Lee, Sidney S. Micek, and 

Allan L. Service, 1977. 



By "educational outcomes" we mean more than the effects of 
postsecondary education on students and the further impacts of 
^^hose on others, including society in general. As used here, 

educational outcomes" refer to any results or consequences of an 
educational institution and its programs. The outcomes may be 
direct results of institutional activities, such as academic 
degrees, technological discoveries, student knowledge and skills, 
or institutional staff salaries. Conversely, there may be later 
consequences of those outcomes, such as individual prestige, 
higher family income, more educated work force, or effects of 
staff salaries on the local economy. 

Educational activities focus on intended outcomes, but 
unintended or unplanned outcomes should also be of concern to 
educators. Unexpected or unintended outcomes can occasionally 
become more important than the intended outcomes. Some unintended 
outcomes are considered to be of positive value by most people 
(for example, increased student ingenuity or creativity), but 
other kinds of outcomes are considered to be detrimental (for 
example, increased drug use or political radicalism). 

Actually, the generic concept of "educational outcome" is a 
neutral one, separated from any inherent value status. But people 
attach value connotations to the outcomes, and even the most 
universally accepted educational outcomes are probably seen as 
negative by some people, [page 1] 

A primary objective of the NCHEMS Outcome Structures and 
Measure project was development of a practical and useful 
classification system that encompasses the full range of outcomes, 
oince any such structure should come out of a conceptual and 
theoretical base, development required some resolution of the 
difficult question of exactly what should be included under the 
rubric outcomes of postsecondary education" and what should be 
excluded. At the present time, the term "outcome" means different 
tnmgs to different people. For some the term denotes "output" 
(Goodman, 1971) or "planned output" (Hoenack et al, 1974). For 
others the same term signifies "end results" or "ultimate 
consequences" (Robinson and Majak, 1967; most PPBS literature). 
fS^n^^^ others the term stands for intended benefits (Hitch, 

li °^ conversely for unintended effects or "side 

effects" (Bauer, 1966; Cook and Scioli, 1972). "Productivity" — 
maximizing outputs obtained from a given amount of resource inputs 
or minimizing inputs needed to produce a given amount of output — 
was the concept of "outcome" emphasized by Hitch (1970) and 
Christenson (1969), while Astin (1970) has focused not on 
no^i'^J^^"'^ outputs with respect to inputs but on comparing output 
conditions, characteristics, and levels to those at input (value 
added). Other frequently ussd synonyms for "outcomes" are 

performance," "efficiency," "effectiveness," and "goals and 
objectives" (that have been achieved) . A survey of the literature 
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on program evaluation and policy analysis clearly indicat. d that 
up until the present time there has been no generally acc-pted 
concept that serves to define the facts known as "outcome i" and t . 
discriminate among the potentially different types or classes of 
outcome measures. Widespread agreement on the need for s«. -ne 
unifying concept has not been sufficient to overcome the 
theoretical complexity that characterizes the issue 'Bartc , 19€ ■ j 
Es^ston, 1965; Goodman, 1971? Schalock et al, 1972? Micek ai ^ 
Wollhaus, 1973). [page 5] 

Traditionally, the programmatic or functional activitie. 
an educational institution and its components that produce ai 
facilitate (or are intended to produce and facilitate) part^ 9. 
outcomes have bee^: divided into three functional areas: 
instructional and socialization activities, research and 
scholarship activities, and public service activities. 
Instruction an'i socialization are the formal (curricular) and 
informal activities provided to help bring about student growth 
and development, that is, knowledge, understanding, competencies, 
attitudes, appreciations, habits, and so forth. Over the years, a 
wide variety of programs and methodologies have been tried in 
different settins to stimulate student growth and development. 

Research and scholarship activities are conducted by units or 
individual staff me.,ibers within the institution with the aim to 
Develop new knowledge or art forms. The new knowledge, 
techniques, or forms of expression ,an be iesigned either to have 
practical application (for example, "applied research") or merely 
to be new (for example, "pure research") . 

Public service activities are those activities that uim 




or facilities for the community. 

It should be noted that these three traditional focuses of 
postsecondary education institutions correspond to the prii"ary 
programs of the NCHEMS Program Classification Structur'- {^CS) , 
outlined in Figure 1. Similarly, the six support programs of the 
PCS (academic support, student service, institutional 
administration, physical plant operations, student financial 
support, and independent operations), could be expected to also 
generate educational outcomes. [page 12] 

^ The structure discusses six attributes or characteristics of 
an educational outcome" plus five other factors important to an 
understanding of this concept. These attributes and factors are 
responses to a number cf questions about outcomes. The questions, 
the attributes and factors associated with each, and what they 
mean are summarized below: 



A. 



What are the characteristics and makeup of an 
"educational outcome"? 
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Form — the basic configuration of the outcome as it is 
observed and/or measured. Outcomes can be separated 
into products, events, and conditions. 

Change Status — Whether the outcome results in 
maintaining (preserving , replenishing , reproducing , or 
stabilizing) or changing (modifying, enriching, 
restructuring, or replacing) the existing condition or 
state of affairs. 

Focus ~ the basic, specific "what" that is maintained 
or changed to constitute the outcome of concern 
(knowledge, understanding, skills, attitudes, roles, 
certification status, jobs, income, social conditions, 
technology, art forms, and so forth). 

Neutrality ~ although people attach positive or 
negative value connotations to specific outcomes, the 
generic concept of "outcome" is a neutral one separated 
from any innerent value status. 

Measurability — the ease with which the outcome can be 
quantified or measured. Some outcomes are easily 
measured? others are difficult to measure. 

Output/ impact ~ Whether there is a direct link between 
the outcome and its producer/facilitator (output) , or an 
indirect link between the outcome and its 
producer/facilitator through outputs and intermediary 
impacts (impact). [page 1 ] 

B. Which institutional resour es and activities are 
combined, and in which way , to bring about the 
outcome (s) of concern? 

Producer/Facilitator ~ the programmatic or functional 
activities of an educational institution or its 
components that produce and facilitate, or ate intended 
to produce and facilitate, particular educational 
outcomes . 

C. For whom is the outcome intended, or who actually 
received or was affected by it? 

Audience — the persons, groups, organizations, 
communities, aggregations of peop?e with common 
observable characteristics, activities or other entities 
that receive and/or are affected by (or are intended to 
receive or be affected by) the outcome of concern. 

D. Why will, or did, the outcome occur? 
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Intended/Unintended — whether the outcome was designed 
or planned to occur or whether it just happened. 
Included are the positive, negative, or neuural value 
connotations attached to an outcome by different people 
and groups, and the "exchange value" perceived for the 
outcome by its producer/facilitator. 

E. Where will, or did, the outcome occur? 

Functional Area — the functional areas within the 
various audience entities that are being affected by (or 
that are meant to be affected by) the outcome, such as 
economic, eriucauional/technological , political, and 
social/cultural/personal . 

F. When will, or did, the outcome occur? 

Time — the time, or expected time, of occurrence of an 
outcome (such as prior to graduation, more than one year 
after graduation) and the duration or persistence of the 
outcome (how long it lasts) . 

Several of these attributes and factors served as a basis for 
dimensions of the Outcomes Structure described in Chapter 3: 
audience, change status, focus, and time. Others have 
implications for using the Outcomes Structure (for exampio. in 
developing lists of outcomes for different cells of the Structure) 
and in analysis of outcomes information. [page 20] 

By definition, a classification system (or structure) for 
outcomes consists of one or more dimensions divided into 
associated categories and subcategories for organizing, 
differentiating, and showing relationships among outcomes. A 
large number of varied attempts have been made using many 
different dimensions to formulate classification systems for 
educational outcomes and such outcome-related concepts as goals 
and objectives. A list of over eighty such classifications found 
by Lenning (19-,7) is in Appendix A. All these cl.issif ications say 
something about ou ;omes and about organizing outcomes, and 
provided useful input to the current NCHEMS effort in this area, 
but several problems are present. Many of the classifications 
found appeared to be quite arbitrary in their content and 
organization, and had their basis in other than emoirical studies. 
Those that are empirically based tend to be narrow' in their area 
of focus, for example, classifying only particular kinds of 
student outcomes. Further, ore , most of the broader 
classifications consist onl, of simple lists of categories, and 
those remaining tend to go :.nto little if any definitional detail. 
None of the broadly focused outcome classification systeps cover 
the full range of potential postsecondary education outcomes. 
Therefore, this new attempt was made to structure outcomes in a 
useful way, and the proposed system is described in this chapter. 
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In certain p. st attempts to structure educational outcomes, 
educators with particular philosophical and theoretical 
orientations have felt that the structure favored other 
philosophies or theories of education and was biased against their 
own beca\:se of the nature of its organization and/or content. 
Therefore, special care was taken to eliminate value and 
philosophical connotations as much as possible from the NCHEMS 
Structure. It is probably impossible to eliminate such bias 
completely, however. 
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Appendix 3 
Compendium of Measures 

1000 Goal Achievement 

1100 Economic Outcomes 

1110 Economic Access and Independence 

1111 Economic Access 

1 2 270 271 272 273 276 

1112 Economic Flexibility, Adaptability, and Security 

3 4 274 275 

1113 Income and Standard of Living 

5 6 

1120 Economic Resources and Costs 

1121 Economic Costs and Efficiency 

7 8 

1122 Economic Resources (including employees) 

9 10 

1130 Economic Production 

1131 Economic Productivity and Production 

11 12 

1132 Economic Services Provided 

13 

1140 Other Economic Outcomes 

1200 Human Characteristic Outcomes 
1210 Aspirations 

1211 Desires, Ainis, and Goals 

14 15 16 253 

1212 Dislikes, Likes, and Interests 

17 18 19 
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1213 Motivation or Drive Level 

20 21 

1214 Other Aspirational Outcomes 



1220 Competence and Skills 

1221 Academic Skills 

22 23 24 

1222 Citizenship and Family Membership Skills 

25 26 27 

1223 Creativity Skills 

28 29 

1224 Expression and Communication Skills 

30 31 32 

1225 Intellectual Skills 

33 34 264 265 

1226 Interpersonal, Leadership & Organizational Skills 

35 36 37 38 

1227 Occupational and Employability Skills 

39 40 41 266 

1228 Physical and Motor Skills 

42 



122 9 Other Skill Outcomes 



1230 Morale, Satisfaction, and Affective Characteristics 

1231 Attitudes and Values 

43 44 45 46 252 254 269 

1232 Beliefs, Commitments and Philosophy of Life 

47 48 49 50 

1233 Feelings and Emotions 

51 52 53 

1234 Mores, Customs and Standards of Conduct 

54 55 56 57 
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1235 Other Affective Outcomes 

1240 Perceptual Characteristics 

1241 PerceptUc->l Awareness and Sensitivity 

58 59 

1242 Perception of Self 

60 61 

1243 Perception of Others 

62 63 

1244 Perception of Things 

64 65 

1245 Other Perceptual Outcomes 

1250 Personality and Personal Coping Characteristics 

1251 Adventurousness and Initiative 

66 67 68 

1252 Autonomy ard Independence 

69 70 

1253 Dependability and Responsibility 

71 72 

1254 Dogmatic/Open-Minded , Authoritarian/Democratic 

73 74 

1255 Flexibility and Adaptability 

75 76 77 

1256 Habits 

78 79 

1257 Psychological Functioning 

80 81 82 

1258 Tolerance and Persistence 

83 84 

1259 Othet Personality and Personal Coping Outcomes 
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1260 Physical and Physiological Characteristics 

1261 Physical Fitness and Traits 

85 86 

1262 Physiological Health 

87 88 

1263 Dther Physical or Physiological Outcomes 

1270 Status, Recognition, and Certification 

1271 Completion or Achievement Award 

89 90 91 92 93 94 

1272 Credit Recognition 

95 96 97 98 

1273 Image , Reputation , or Status 

99 100 101 102 103 

1274 Licensing and Certification 
104 105 106 107 198 

1275 Obtaining a Job or Admission to a Follow-up Pgm 
108 109 110 111 112 

1276 Power and/or Authority 
113 114 115 116 

1277 Job, School, or Life Success 
]17 118 119 

1278 Other Status, Recognition&Certif ication Outcomes 

1280 Social Activities and Roles 

1281 Adjustment to Retirement 
120 121 

1282 Affiliations 
122 123 

1283 Avocational and Social Activities and Roles 
124 125 
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1284 Career and Vocational Activities and Roles 
126 127 277 

1285 Citizenship Activities and Roles 
128 129 130 

1286 Far^y Activities and Roles 
131 132 

1287 Friendships and Relationships 
133 134 

1288 Other Activity and Role Outcomes 

1290 Other Human Characteristic Outcomes 

1300 Knowledge, Technology, and Art Form Outcomes 

1310 General Knowledge and Understanding (K & U) 

1311 K & U of General Facts & Terminology 
135 136 

1312 K & U of General Processes 
137 138 

1313 K & U of General Theory 
139 140 

1314 Other General Knowledge & Understanding 

1320 Specialized Knowledge and Understanding (K s> U) 

1321 K & U of Specialized Facts & Terminology 
141 142 250 251 

1322 K & U of Specialized Processes 
143 144 199 251 

1323 K & U of Specialized Theory 
145 146 251 

1324 Other Specialized Knowledge & Understanding 
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1330 Researcn and Scholarship 

1331 Research & Scholarship Knowledge & Understandi 
147 148 200 249 

1332 Research and Scholarship Products 
149 150 

1340 Art Forms and Works 

1341 Architecture 
151 152 

1342 Dance 
153 

1343 Debate and Oratory 
154 155 

1344 Drama 
156 157 

1345 Literature and Writing 
158 159 

1346 Music 
160 161 

1347 Painting, Drawing, and Photography 
162 163 

1348 Sculpture 
164 165 

1349 Other Fine Arts 



1350 Other Knowledge, Technology and Art Form Outcomes 
1400 Resource and Service Provision Outcomes 
1410 Provision of Facilities and Events 

1411 Provision of Facilities 
166 167 284 

1412 Provision or Sponsorship of Events 
168 169 
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1420 Provision of Direct Services 

1421 Teaching 

170 278 

1422 Advisory and Analytic Assistance 

171 172 

1423 Treatment, Care, and Referral Services 
173 174 

1424 Provision of Other Services 
175 280 281 282 

14J0 Other Resource and Service Provision Outcomes 



1500 Other Maintenance and Change Outcomes 

1510 Aesthetic-Cultural Activities, Traditions & Conditions 



1520 Organizational Format, Activity and Operation 



176 



177 



178 



1530 Other Maintenance and Change 



2000 Managerial Processes 



2100 Planning 



330 



331 332 333 334 358 



2200 Organizing 



337 



338 339 340 341 342 



2300 Directing 



347 



351 352 353 354 355 356 357 359 360 



2400 Control 



335 



336 



2500 Staffing 



343 



344 345 346 348 349 350 
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3000 Organizational Climate 



3100 Autonomy 
405 

3200 Structure 
405 

3300 Consideration and Support 
405 

3400 Synergy 

354 405 

3500 Reward Orientation 
352 353 405 

3600 Openness vs. Def ensiveness 

355 405 

3700 Participation vs. Decision Centraliratior, 
351 405 

3800 Educational Climate 
404 

4000 Environmental Adaptation 



4100 Efficiency 

193 226 227 228 229 234 235 243 244 245 216 247 
248 258 259 

4200 Productivity 

194 195 198 200 217 234 255 256 257 264 293 294 
295 305 391 

4300 Bargaining Position 



431C Tangible Resources 



4311 Physical Plant 

224 236 237 238 239 242 310 311 312 374 399 
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4312 Support Services 
132 196 197 



4313 Facilities 
201 

4314 Staff 

225 230 231 232 233 298 299 300 301 302 303 304 
324 378 398 

4315 Students 

180 181 182 184 185 186 187 188 189 190 202 203 
204 205 206 207 261 262 279 293 294 295 297 305 
323 325 326 328 365 367 369 372 376 377 403 183 

4316 Finances 

208 209 210 211 212 213 214 215 216 217 218 21^'> 

220 221 222 223 296 306 307 308 309 310 312 313 

314 315 316 317 318 319 320 321 322 329 372 373 

375 378 381 382 384 385 386 392 393 394 401 402 
406 407 408 409 410 

4317 Research 
400 

4318 Educational Programs 

219 327 328 379 383 387 390 391 395 396 397 

4320 Intangible Resources 

196 197 218 260 280 281 282 283 285 286 287 288 
289 290 291 292 323 325 326 374 

4400 Client Satisfaction 

182 190 191 193 260 263 265 266 269 380 390 391 

4500 Capacity to Test Reality 
267 

4600 Control Over The Environment 

213 214 215 263 303 371 373 406 407 408 409 410 

4700 Flexibility/Readiness 



213 


214 


215 


231 


232 


233 


234 


235 


238 


239 


240 


241 


244 


245 


246 


247 


248 


296 


297 


298 


299 


300 


301 


302 


303 


306 


307 


308 


309 


310 


311 


312 


313 


314 


315 


316 


317 


318 


319 


320 


321 


32? 


323 


324 


325 


326 


327 


328 


329 


361 


362 


363 


364 


365 


366 


367 


368 


369 


370 


371 


373 


375 


377 


379 


380 


381 


382 


384 


385 


386 


387 


388 


389 


392 


393 


394 


406 


407 


408 


409 


410 
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Appendix 4 

Measures and Indicators of Organizational Effectiveness 



Indicator 

Indicator 



1 Percentage of students obtaining their first FT job in the field 
of their choice within a specified time after graduation. 

2 Number of alternatives for an entry level job open to minority 
group graduates compared to minority group nongraduates . 

3 Geographic mobility of college graduates compared to those not 
attending college. 

4 Self-report of college graduates about the economic security for 
them and their families, and the contribution of college to this. 

5 Amount of annual and lifetime earnings of those attending college 
compared to those not attending college. 

6 Average student and/or former student reported score, on scales 
measuring perceptions and evaluations of their current and desired 
socio-economic level. 

7 The absenteeism and tardiness on-the-job of college graduates as 
compared to nonstudents. 

8 The number of firms that use the college degree as an inexpensive 
screening device that allows them to hire qualified employees at 
minimum initial cost to the firm. 

9 Percentage of college graduates employed in management positions 
within a specified time after graduation. 

10 Average number of patents and/or copyrights received per student, 
former student, and/or faculty member. 

11 Percentage of college graduates who can adequately do their 
personal typing and complete their own income tax forms as a 
result of having attended college. 

12 Expert judges' ratings of the amount of increased worker 
production and higher worker motivation that results from having 
attended college. 

13 Dollar amount of goods and services bought in the local community 
by the institution, its staff, and its students. Number of hours 
of consultation in the business area provided to area companies 
and institutions by the university's college of business. 
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14 Changes in observed desires from college entrance to graduation. 

15 Changes in the reported aspirations for graduate schoo] as a class 
proceeds through undergraduate school. 

16 Self-report of changes m goals and aspirations as a result of 
college . 

17 The reported likes and dislikes of persons before college as 
compared to after graduation, and comparison with such change over 
the same period of time for those the same age not attending 
college. ^ 

18 Score or change in score on an interest inventory, e.g.. Strong 
Vocational Interest Blank, Kuder General Interest Survey, Kuder 
Occupational interest Survey, ACT interest inventory. 

19 Self-report of changes in interests as a result of college. 

20 Score or change in score on an instrument that measures "need for 
achievement" or "achievement motivation," e.g., the Strong 
Vocational Interest Blank Academic Achievement Scale, the Colleoe 
Student Questionnaire Motivation for Grades Scale, the California 
Psychological Inventory Achievement Scales, Personal Value 
Inventory . 

21 Self-report of changes in motivation level as a result of college. 

22 Grades earned when the effect of ability, motivation, and other 
such factors have been controlled. 

23 Persistence in college when the effects of ability, motivation, 
and other such factors have been cancelled out. 

24 Score or change in score on a test of study skills, e.g.. 

Brown- .loltzman Survey of Study Habits and Attitudes, Comprehensive 
Test of Basic Study Skills. 

25 Self-report of abilities pertaining specifically to citizenship 
and home membership that college accentuated. 

26 Evaluation by others of citizenship and home membership skills 
mastery exhibited. 

27 Score or change in score on the Vinelanc Social Maturity Scales. 

28 Score or change in score on a test that measures oriainalitv and 
creative ability, e.g., Minnesota Test of Creative Thinking", Test 
of Creative Ability, Guilford's Alternate Uses Test, Sixtepn 
Personality Factors Questionnaire Creativity Scale. 
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9 Evaluation by judges of creetive ability demonstrated in a 
building or forming task. 



30 



Score or change in score on tests that measure the ability .o 
communicate or express oneself. 



31 Judges' rating in a debate or speech contest. 

32 Judges' rating of expression in a music, art, or ballet contest. 

33 Score or change in score on a test that measures ability to 
analyze and solve problems and to make inferences, e.g., 
California Test of Mental Maturity, Watson-Glazer Critical Thinking 
Appraisal, Californii Psychological Inventory Intellectual 
Efficiency Scale. 

34 Self-report of changes in analytical ability as a result of college. 

35 Leadership awards. 

36 Self-perceptions and evaluation of interpersonal and leadership 
ability. 

37 Perceptions by judges of interpersonal and leadership skills. 

38 Score or change in score on a test that measures leadership and 
••nterpersonal ability, e.g., California Psychological Inventory 

adership Scale, Chapin Social Insight Scale. 

39 Spatial relations test scores for someone who is, or is going to 
be, an artist. 

40 Demonstrated ability in writing FORTRAN or COBOL for someone who 
IS, or is going to be, a computer programmer. 

41 Score or change in score on the Bennett Mechanical Comprehension 
Test. 



2 Judges' scores on skill events in athletic competition such as 
gymnastics, diving, and figure skating. 

3 Score or change in score on an attitude scale, e.g., Thurstono and 
Chave's Scale for Measuring Attitudes Toward the Church, College 
Student Questionnaire Part I, Adorno Ethnocentrism Scale, Shaw and 
Write Scales for the Measurement of Attitudes. 

4 Self-report of one's attitudes and the effect of college on them. 
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Score or change in score on an instrument that assesses values, 
e.g., Alport-Vernon-Lindsey Study of Values, Differential Value 
Profile, Work Values Inventory. 

Self-report of one's values and the effect of college on helping 
to clarify them. 

Score or change in score on instruments that assess beliefs, e.g., 
Harvey's Conceptual Systems Test, inventory of Beliefs. 

Self-report of one's beliefs and commitments and the effect of 
college on their. 

The membership and participation in, and support of, a particular 
religious organization or cause prior to as compared with after 
col lege . 

Self-report of one's philosophy of life and the effect of college 
on clarifying and organizing it. 

Openness and acceptance of feelings before college compared to 
after college. 

Development of an appreciation of different cultures and a wide 
range of human values as a result of college. 

Greater reported satisfaction with life as a result of college. 

Self-report of the effect of college on assimilation or 
internalization of the customs of community or society. 

Score or change in score on the California Psychological Inventory 
Socialization Scale. 

The adherence to particular mores or social customs prior to 
college as compared to after college. 

The amount of subjectivity and emotion guiding one's standards of 
conduct prior to college as compared to after college. 

Increased sensitivity to needs and emotional cues provided by 
others. 

increased alertness to the opportunities confronting one. 

Development of positive self-regard and self-confidence as a 
result of college. 
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61 Score or change in score on a self-concept scale, e.g., Adjective 
Check List; California Psychological Inventory Self Acceptance 
Scale, Tennessee Self Concept Scale. 

Reports by observers about how a person's respect for others has 
changed as a result of collec>e. 

Self-report of how one's view of ochers has changed as a result of 
college . 



62 
63 
64 



66 
67 
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Increased respect for the ideas of others as result of college. 

65 Movement as a result of college experiences from seeing things as 
all "black and white" to complex "grays". 

Reports by impartial observers of changes in initiative that seem 
to have resulted from college attendance. 

Self-report of the effect of college on one's willingness to take 
a chance, e.g., to take an educated guess on an exam. 

68 The frequency that one exhibits speaking out on issues as the 
college career progresses. 



Score or change in score on personality scales that measure 
autonomy and independent, e.g., Sixteen Personality Factors 
Questionnaire Group-Dependent vs. Sel f-Suf f icient Scale, Edwards 
Personality Inventory Independent in His Opinions Scale, College 
Student Questionnaire Independence Scales, Omnibus Personality 
Inventory Autonomy Scale. 

Self-report of willingness to volunteer or "stand up for one's 
rights" and the effect of college attendance on such willingness. 

71 Reports by observers of changes in dependability and 
responsxbility that have occurred during college. 

72 Score or change in score on scales that measure dependability and 
responsibility, e.g., California Psychological Inventory 
Responsibility Scale, Edwards Personality Inventory Assumes 
Responsibility Scale, Sixteen Personality Factors Questionnaire 
Expedient vs. Conscientious Scale. 

73 Reports of expert observers about changes in open-mindedness that 
have taken place during college. 

74 Score or change in score on e scale that measures dogmatism and/or 
authoritarianism, e.g., Rokeach Dogmatism Scale, California 
Psychological Inventory Dominance Scale, Omnibus Personality 
Inventory Religious Orientation Scale. 
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Score or change in score on a scale that measure flexibility, 
e.g., California Psychological Inventory Flexibility Scale, 
Omnibus Personality Inventory Practical Outlook Scale, Sixteen 
Personality Factors Questionnaire Practical vs. Imaginative Scale. 

Reports by observers of changes in adaptability and flexibility 
that have occurred during college. 

Self-report of the effect of college on adaptability and 
flexibility. 

Observations by others of changes in habit orientation that have 
occurred during attendance. 

Self-report of changes in habits that have resulted from college. 

The amount of realization of one's actual strengths and 
weaknesses, and of what is reality. 

Score or changes in score on an instrument that measures 
psychological adjustment, e.g., Minnesota Multiphasic Personality 
Inventory, Sixteen Personality Factors Questionnaire, Moody 
ProDlem Check List. 



Reports by expert observers about changes in the psychological 
functioning of individuals that have occurred during college 
attendance. 

Observations by others of changes in tolerance and persistence 
during college. 

Score or changes in score on an instrument that measures tolerance 
and persistence, e.g., Edwards Personality Inventory Persistence 
Scale, California Psychological Inventory Tolerance Scale. 

Score or change in score on physical fitness tests, e.g., AAHPER 
Youth Fitness Tests, Basic Fitness Tests. 

Self-report of "feeling in better physical shape" as a result of 
college. 

Medical doctor's health physical examination report at college 
entrance compared to at college graduation. 

Self-report of the effect of college attendance on how well alumni 
take care of their bodies. 
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89 An honorary degree. 

90 Graduation diploma. 

91 AlUinni achievement award. 

92 Sales avard or a job promotion. 

93 Danforth Fellowship Award. 

94 Being named a Rhodes Scnolar. 

95 Graduate school grades. 

96 Credit hours given for completing a course. 

97 By-line credit for a movie, play, book, or article. 

98 Financial credit rating issued by a bank or credit bureiiu. 

99 Being on the social register. 

100 Being listed in Who's Who. 

101 Oral and written acknowledgments from others. 

102 Being interviewed by the press, radio, or TV. 

103 Writing an autobiography that is published or having a biography 
written about you. 

104 Entry into the state bar. 

105 Passing a cosmetology licensing exam. 

106 Being a certified public accountant. 

107 An insurance company that has been licensed to sell in a state. 

108 Entrance to a university after graduation from a community college 

109 Entrance to law, medical, or graduate school. 

110 Being selected by the civil service. 

111 Peing selected for a company executive position. 
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112 Being hired in the special field for which the training applied. 

113 Appointment or election to a position of authority. 

114 Earning promotion to a position of authority. 

115 Influencing important community or public decisions. 

116 Getting acknowledged credit for the important job having gotten 
done. 

117 Self-report of success in career. 

118 Teacher's rating of success in graduate school. 

119 Employer's rating of overall on-the-job performance. 

Percentage of college education retirees reporting productive 
retirement years compared to reports of those who never attended 
college . 



120 



121 Self-report of the effect of having attended college on the 
retirement years. 

122 Number of affiliations and changes in affiliations for college 
graduates as compared to those never attending college. 

123 Self-report of the effect of having attended college on the 
affiliations sought and on the affiliations won. 

124 The social roles and avocations of college graduates as compared 
to those who never attended college. 

125 Self-report of the effect of having attended college on the 
avocational and social roles sought, and on those practices. 

126 The career roles of college grad>jates as compared to those who 
never attended college. 

127 Reports of employers concerning the advancement and roles of 
college trained employees versus the advancement and occupational 
roles of those who never attended college. 

128 Percent voting in a municipal or state election. 

129 Financial and other contributions given to service organizations. 
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Percent running for public office or campaigning for someone who is. 



131 The family roles of college graduates as compared to those who 
never attended college. 

132 Self-report of effect of the college on the roles played in one's 
family. 

133 Characteristics of friends and relationships of college educated 
people versus those never attending college. 

134 Self-report of the effect of college on friendships and social 
relationships. 

135 Students' scores or chcnooc in score on standardized or classroom 
tests that measure knowledge and understanding of general 
terminology and/or facts. For example, the Miller Analogies Test 
focuses entirely on knowledge and understanding of general 
terminology, and tests like the College Level Examination Program 
(CLEP) or the Graduate Record Exam (GRE) general exam include 
coverage of general terminology and facts. 

136 Students' self-report of knowledge and understanding about general 
terminology and facts. 

137 Students' scores or changes in score on standardized or classroom 
tests measuring comprehension of general conventions, processes, 
and methodologies. 

138 Students' grades in a general application survey course. 

139 Students' scores or changes in score on standardized or classroom 
tests measuring comprehension of general theories in a broad field 
of study. 
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Students' grades in a general survey course on theories of 
philosophy. 



141 Students scores or changes in score on standardized or classroom 
^ests that measure knowledge and understanding in a narrow, 
specialized area of study. Professional certification and 
licensing exams usually focus on this type of knowledge, as do 
tests liki the College Level Examination Program (CLEP) subject 
exams or the Gradu?*-'; Record Exam (GRE) area exams. 

142 Students' self-report of knowledge and understanding abouv 
special i7«ri terrr-inclcgy 5r.<3 Ta^Ls,. 
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143 Students scores or changes in score on standardized or classroom 
tests measuring comprehension of conventions, processes, 
methodologies, and techniques unique to particular specialized 
professions or disciplines. 

144 Students' grades in a sprcializec professional course or program. 

145 Students' scores or changes in score on standardized or classroom 
tests measuring comprehension of specie lized theoretical 
formulations and models. 

146 Students' grades in a course that goes into depth about one or 
more theories or models unique to a specialized discipline or 
profession. 

147 Average number of basic research publications, applied research 
publications, textbooks, or monographs, etc., per student, former 
student, and/or faculty member over a specif ic' period of time. 

148 Number of faculty members and/or former students in the sciences 
listed in American Men of Science. 

149 Average number of patents and/or copyrights received per student, 
former student, and/or faculty member over a given period of time. 

150 Average number of awards and citations received per student, 
former student, and/or faculty member (over a given period of 
time) for discovery of development of technological products. 

151 Number of architectural works completed by students, former 
students, and/or faculty. 

152 Number of awards and other recognitions received for architectural 
works on the campus commissioned by campus officials. 

153 Number of students involved in dance auditions and public 
perlormances . 

154 Competition record over a period of years of the college's debate 
team. 

155 The average number of graduates each year who go on to some kind 
of oratorical career. 

156 The number of students who enter a professional acting career, and 
the number acting on an amateur bfl<;i<;- 
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157 The number of drama performances put on for the local community 
each year. 

158 The average number of literary works each year published by 
students, former students, and/or faculty members. 

159 The number of students and faculty each year who have entered a 
formal state or national writing competition. 

160 The number of musical productions put on each year by the college 
that are open to the public. 

161 The number of students involved public music recitals ard other 
performances . 

162 The number of paintings, and their quality in the campus art 
gallery. 

163 The number of awards won over a certain period of time for 
pictorial works by students, former students, and faculty members. 

164 The number of sculptures that have been commissioned by the 
college and placed throughout the campus. 

165 The forms of sculpture that have been developed on the campus. 

166 Number of facilities made available to the students during a 
particular period of time. 

167 Total number of hours each facility was used by people in the 
community, and the number of people-nours of use over a specific 
period of time. 

168 The number of people who attended athletic events, cultural 
events, or other events provided and/or sponsored by the college 
in any one year. 

169 The number of column inches of newspaper coverage received by 
specific events in local, regional, and national newspapers. 

170 Average number of courses taught and number of contact hours per 
semester in the regular program. Extension courses provided in 
any one calendar year. 

171 Numbr:r of advisory and analytic assistance services offered to 

students, staff. SnH/nr *-ry <-V^rs .->..ui^_ 
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172 Number cf persoii-hours spent by staff in providing this assistance 
over a specific period of time. 

173 The treatment, care, and referral se'-vices offered by the 
institution and its staff, and health services, day care for 
children of working mothers, counseling, crisis referral, and drug 
treatment and the amount these services are used. 

174 The reported satisfaction of users of these services with the 
treatment and care received. 

175 Provision of Other Services— An example would be direct civic 
leadership provided to the community. Another example would be 
offering keypunching service. 

176 Other Resource and Service Provision Outcomes— an example would 
be the attention and good will the college draws to the local 
community because it is located there. 

177 Aesthetic-Cultural Conditions— Preserving or bringing about 
changes in tastes, level and kinds of aesthetic-cultural emphasis, 
aesthetic-cultural availability and opportunities, 
aesthetic-culture activity and participation, etc. 

178 Organizational Format, Activity, and Operations — For 
organizations, groups, and systems (and their components), 
maintenance or change in organizational communications, 
operational methods and interaction, operational effectiveness, 
organizational relationships, organizational arrangement and 
configuration, organizational activities and programs, and other 
such organizational characteristic outcomes. 

179 Other Maintenance and Change — Outcomes not covered by any of the 
other subcategories of "Maintenance" and "Change" in this 
dimension of the Outcomes Structure. An example might be 
"destruction of life support in the environment". 
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Potential Applicants to Postsecondary Education: The estimated 
number of individuals who might be enrolled in (i.e., participate 
in) various postsecondary education programs and/or activities if 
available in a particular region within a specified time period. 

Applicants to Postsecondary Education Programs and/or ^^.ctivities : 
The number of different individuals actually makinc, applications 
(institutionally dpfir»e<^) for sdriiiGsion inLu t^arcicuiar 
postsecondary education programs and/or activities in a particular 
region within a specified time period, without regard to available 
openings or the applicants' qualifications. 



133 



182 Postsecondary Education Applicants Accepted: The total number of 
individual applicants for admission to a particular postsecondary 
education institution and/or program or activity to whom formal 
notification of acceptance is given, within a specified time 
period. 

183 High School Graduates: The number of students who satisfactorily 
complete (i.e., graduate from) a secondary education program in a 
particular region within a specified time period, usually an 
academic year. 

184 High School Graduates - Continuing to Postsecondary Education: 
The number of "High School Graduates" [2070] who continue their 
education in a postsecondary education program or activity in a 
particular region within a specified time period. 



185 First-time Entering Students - Undergraduate: The (headcount) 
number of students who enter a particular postsecondary education 
reporting unit at the undergraduate level for the first time, 
regardless of student loads, with less than one semester (or 
semester equivalent) of academic credit earned at another 
reporting unit which is applicable Cor credit at the reporting 
unit of current enrollment, within a specified time period. 

186 First-time Entering Students - First Professional: The 
(headcount) number of students who enter a particular 
postsecondary education reporting unit at the first professional 
level for the first time, regardless of student loads, within a 
specified time period. 

187 First-time Entering Students - Graduate: The (headcount) number 
of students who enter a particular postsecondary education 
reporting unit at the graduate level for the first time, 
regardless of student loads, within a specified time period. 

188 Transfer Students: The (headcount) number of students enrolled in 
a particular reporting unit for the first time with one semester 
credit (or semester credit equivalent) or more of academic credit 
earned at another postsecondary education institution which is 
applicable for credit at the current reporting unit in a program 
or course of study at the same program level, within a specified 
time period. 



189 Continuing Students: The (headcount) number of students enrol Ipd 
ir^ a particular i.et"-'!. iing unit at a defined subsequent regular 
session who are continuing toward their program objective, 
regardless of student levels or program levels within a specified 
time period. 
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190 Readmitted Students: The (headcount) number of students who 
withdraw from a particular reporting unit and later return and 
enroll to continue or complete a postsecondary education program 
^^^^^V'^^® °^ study, and do not qualify as "Continuing Students" 
i2160], within a specified time period. 

191 Discontinuing Students: The (headcount) number of students 
enrolled in one regula- session and not the next at a particular 
reporting unit within a specified time period. 

192 Financial Aid - Total: The total dollars make available to all 
students at a particular reporting unit for financial assistance 
regardless of the source (s) of funds within a specified time 
period . 

193 Average Incurred Cost to Student - Out-of-Pocket: The average 
out-of-pocket costs incurred by a student enrolled at a particular 
reporting unit in a specific program and/or activity within a 
specified time period, usually an academic year. 

194 Instruction Programs, Courses of Study, and Activities - 
Completions: The total number of students who complete or attain 
the objective (s) of each of the instruction programs, courses of 
study, and activities at a particular reporting unit within a 
specified time period. 

195 Research Programs and Activities: An inventory of the names of 
all the research programs and activities at a particular reporting 
unit established to produce distinct research outcomes 
commissioned by an agency either external to the reporting unit or 
authorized by an organizational unit that are in progress at the 
reporting unit within a specified time period. 

196 Public Service Programs and Activities: An inventory of the names 
ot all the public service programs and activities to which a 
particular reporting unit makes available resources to produce 
outputs and services that are directed toward the benefit of the 
community, or individuals residing in the region servecj by the 
institution, within a specified time period. 

197 Public Service Programs and Activities - Completions: The total 
number of public service programs and activities completed at a 
particular reporting unit within a specified time period. 

198 Certification and Licensing Examinations - Attempts: The total 
number of students (both current and former) of a particular 
reporting unit who try to pass state, regional, and/or national 
agency certification and licensing examinations within a specified 
time period. ^ 
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199 Certification and Licensing Examinations - Successful 

Completions: The total number of students (both current and 
former) of a reportino unit who pass state, regional, and/or 
national agency certification and licensing examinations within a 
specified time period. 
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Sponsored Project Proposals Submitted: An inventory of the names 
of all the sponsored project proposals (i.e., requests for 
funding) officially submitted by a particular institution or 
agents of the institution, to some external funding body for their 
consideration, within a specified time period. 




202 Student Enrollment - Headcount: The unduplicated count of the 
number of persons at a particular reporting unit who are enrolled 
in postsecondary education courses of study, programs, and 
activities witnin a specified time period. 

203 Full-time Students: The (headcount) number of students, 
regardless of student level (s), courses of study, programs, or 
activities, who are enrolled for (i.e., registered for) at least 
75 percent of the normal student load required to complete the 
student's program of study (whether for credit, noncredit, imouted 
credit, contact hours, etc.) within the normal time to complej-e a 
student program, course of study, or activity in a particular 
reporting unit at a specified point in time. 

204 Part-time Students: The (hoadcount) number of students regardless 
of student level (s), courses of study, programs, or activities who 
are enrolled for (i.e., registered for) less than 75 percent of 
the normal student load required to complete the student's program 
of study (whether for credit, noncredit, imputed credit, contact 
hours, etc.) within the normal time to complete a student program, 
course of study, or activity in a p..rticular reporting unit at a 
specified point in time. 

205 Full-time Equivalent Sturt^^nts: » ctudent couuL calculated by 
dividing the total number of student load units generated at a 
particular reporting unit by a standard student load measure for a 
specified time period (e.g., quarter, semester, academic year, 
fiscal year, etc.) 
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206 In-state Students: The (headcount) number of students who attend 
SkF!k iJ" postsecondary education institution in the state in 
which they legally reside at a specified point in time. 

207 Out-of-state Students: The (headcount) number of students who 
attend a particular postsecondary education institution that is 
outside of the state in which they legally reside at a specified 
point in time. 

208 Total Assets - All Fund Groups: The total dollar values (i.e., 
wh?^h^jK'^'''' '"Kf^^^ values) of the property cf all fund groups to 
which the right of ownership, possession, and/or legal title have 
been assumed by a particular institution at a specified point in 

209 Total Liabilities - All Fund Groups: The total dollar values 

and/or n.^.^et values) of the claims against assets, 
money owed, and debts on pecuniary obligations (i.e., liabilities) 
of all fund groups of a particular institution at a specif i»H 
point in time which require settlement in the future. 

210 Total Fund E Uances - Al ' Fund Groups: The total dollar value 
(I.e., hook and/or marltet values) of the fund balances of all fund 
groups of a particular institution at a specified point in time. 

211 Total Additions - All Fund Groups: The total dollar value of any 
additions to institutional resources in any fund groups of a 
particular institution within a specifieJ time period. 

212 Total Deductions - All Fund Groups: The total dollar value of any 
deductions of institutional resources from any fund groups of a 
particular institution within a specified time period. 

213 Total Net Change in Fund Beiances - All Fund Groups: The total 
dollar value of the next changes of the fund balances of all fund 
groups of a particular institution within a specific-1 time 
period. 

214 Total Current Funds Revenues: The total dollar amount of all 
unrestricted funds from gifts and dther resources earned during 

i"*^.^""^ restricted funds to the extent 

that such funds ware expended for current postsecondary education 
program and activity operating purposes from the current funds by 
a particular institution, within a specified time period, usually 
a fiscal year. ^ 

l^l^l Curren-. Funds Expenditures: The total dollars expended from 
the current funds, by a particular institution, omitting only 
depreciation, to support postsecondary education programs and 
activities within a specified time period, usual if a fiscal yet-r. 
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216 Instruction Expenditures: The total dollars expended from the 
current f-nds for those program elements (e.g., courses, 
activities, work experiences, etc.) whose outputs may be eligible 
for credit in meeting specified formal curricular requirements, 
leaaing toward a particular postsecondary education 
degree/diploma/certificate granted by as particular institution 
within a specified time period, regardless of source of funding. 

217 Research Expenditures: The total dollars expended from the 
current funds for program elements (e.g., projects, activities, 
work experiences, etc.) that have been specifically organized to 
produce research outcomes commissioned by an agency either 
externa] to a particular institution or authorized by an 
organizational unit in the institution within a specified time 
period, regardless of source of funding. 

218 Public Service Expenditures: The total dollars expended from the 
current funds for program elements which are established to make 
available to the publir the various resources and capabilities of 
a particular j-notitution within a specified time period, 
regardless of source of funding. 

219 Academic Support Expenditures: The total dollars expended from 
the current funds for all program elements carried out primarily 
to provide support services that are an integral part of the 
operations of instruction, research, and public service programs 
at ? particular institution within a specified time period, 
regardless of source of funding. 

220 Student Services Expenditures: The total dollars expended from 
the current funds for all program elements whose primary purpose 
IS to contribute to the students' emotional and physical 
well-being and to their intellectual, vocational, cultural, and 
social development outside the context of the formal in truction 
program at a particular institution within a specified time 
period, regardless of source of funding. 

221 Institutional Support Expenditures; The total dollars expended 
from the current funds for all program elements whose primal^, 
purpose IS to maintain the organizational effectiveness and 
continuity of a particular institution within a specified time 
period, regardless of source of funding. 

222 Independent Operations Expenditures: The total dollars expended 
from the current funds for all operations which are independent 
of, or unrelated to, the primary missions of a particular 
institution (i.e., instruction, research, and public service), 
although they may contribute indirectly to the enhancement of 
these progr>?ms, within a specified time period, regardless of 
source of funding. 
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223 Scholarship and Fellowship Expenditures; The total dollars 
expended from the current funds given to individuals enrolled in 
formal course work (whether for credit or not) in the form of 
outright grants and trainee stipends, at a particular institution 
within a specified time period, regardless of source of funding. 

224 Capital Asset Expenditures: The total dollars expended from all 
fund groups (e.g., current funds and plant funds) for land, 
improvements to land, buildings, additions to buildings, and 
capital equipment at a particular institution within a specified 
time period, regardless of source of funding. 

225 Compensation Expenditures: The total dollars expanded from the 
current funds group for direct or indirect compensation to all 
employees of a particular reporting unit within a specified time 
period, regardless of source cf funding. 

226 Full Cost per Semester Credits: The sum of direct costs, capital 
costs, and allocated support costs assigned to a set of 
instructional activities divided by the total semester credits or 
semester credit equivalents generated by those instructional 
activities at a particular reporting unit within a specified ti.ne 
period. 

227 Full Cost per Contact Hours: The sum of direct costs, capital 
c sts, and allocated support costs assigned to a set of 
instructional activities divided by the total student contact 
hours generated by those instructional activities at a particular 
reporting unit within a specified time period. 

228 Full Cost per Course Enrollments: The sum of direct costs, 
capital costs, and allocated support costs assigned to a set of 
instructional activities divided by the total course enrollments in 
those instructional activities at a particular reporting unit 
within a specified time period. 

229 Full Cost per Full-time Equivalent Student: The sum of direct 
costs, capital costs, and allocated support costs assigned to a 
^et of instructional activities divided by the total number of 
full-time equivalent students engaged in those instructional 
activities at a particular leporcing unit within a specified time 
period . 

230 Staff - Headcount: The unduplicated count of the number of 
individuals employed at a particular reporting unit at a specified 
point in time. *^ 
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231 Full-time Staff: The (headcount) number of staff who are employed 
under a regular full-time contract/appointment/agreement at a 
particular reporting unit, or its designated agent, at a specified 
point in time. 

232 Part-time Staff: The (headcount j number of staff employed at a 
particular reporting unit or its designated agent, who are not 
under a regular full-time contract/appointment/agreement at a 
specified point in time. 

233 Full-time Equivalent Staff: A staff count calculated by dividing 
the total number of occupational workload units (e.g., hours, 
courses and/or activities taught, etc.) generated at a particular 
reporting unit by a standard occupational workload measure within 
a specified time period. 

f^f^^/f*'^^^^^ Workload - Average: "Full-time Equivalent Staff" 
17100] divided into the total number of hours spent by them in 
various employment activities at a particular reporting unit 
within a specified time period. 

235 Student - Faculty Ratio: "Full-time Equivalent Student" [5100] 
divided by the number of full-time equivalent faculty at a 
particular reporting unit within a specified time period. 

236 Land Area: The total land surface owned, rented, leased, or 
otherwise under the control of a particular reporting unit, at a 
specified point in time. 

237 Gross Area: The total area of buildings included within the 
boundaries of a particular reporting unit, at a specified point in 
time . ^ 

238 Assignable Area: The total area on all floors of a building at a 
particu.ar reporting unit assiyned to, or available for assignment 
to, an occupant; including every type of space functionally usable 
by an occupant, at a specified point in time. This excludes areas 
Area" °8070p''^ operation of a building (i.e., "Nonassignable 

239 Nonassignable Area: The total area on all floors of a building at 
a particular reporting unit that is not available for assignment to 
building occupants, but that is necessary for the general 
operation of the building, at a specified point in time. 

240 Enrollment Capacity: The number of students and other 
participants that can be accommodated in programs, courses of 
study, and activities of a particular reporting unit, at a 
specified point in time. 
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241 Design Capacity: The total number of stations which all of the 
rooms of a building or physical facility at a particular reporting 
unit are designed to accommodate when used in the manner currently 
intended, at a specified point in time. 

242 Estimated Replacement Value: The total estimated cost to replace 
the assignable floor area of buildings at a particular reporting 
unit at current construction costs, in accordance with current 
building codes, standard construction methods, and currently 
accepted policies and practices, at a specified point in time. 

243 Average Section Size (AvSS) : The average number of students in 
the sections of a group of programs, courses of study, and/or 
activities at a particular reporting unit, within a specified time 
period. 

244 Average Square Feet per Weekly Student Hour (AvSFWSH) : The 
average number of assignable square feet (ASF) divided by the 
number of regularly scheduled weekly student hours (WSH) in a room 
or group of rooms at a particular reporting unit, within a 
specified time period. 

245 Average Room Utilization Rate (AvRUR) : The average number of 
houro per week that a room or group of rooms at a particular 
reporting unit is scheduled for use, within a specified time 
period . 

246 Average Station Occupancy Ratio (AvSOR) : The ratio of the average 
proportion of stations utilized divided by the stations available 
for use, for all lengths of time when a room or group of rooms at a 
particular reporting unit is scheduled for use, within a specified 
time period. 

247 Average Station Utilization Rate (AvSUR): The average number of 
hours per week that the stations in a roon, oi group of rooms at a 
particular reporting unit are scheduled for use, within a 
specified time period. 

248 Occupancy Rate: The actual number of occupants of a facility at a 
particular reporting unit divided by the facility's "Design 
Capacity" [8140], within a specified time period. 

249 Student development concerning breadth of knowledge: Student 
scores on tests that indicate development in their breadth of 
knowledge about facts and principles across several broad fields 
of study (the humanities, the physical sciences, etc.). 
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student development concerning d(?pth of knowledge: Student scores 
on tests that indicate development in their depth of knowledge 
concerning facts and principles in the particular fields in which 
students elect to study. 

Student success in passing certification and licensing 
examinations: Number and percentage of students and/or former 
students passing certification and licensing examinations (e.g., 
Bar Lxam, CPA Exam, LPN Exam) . 

Areas and agents of student change during college: Student scores 
on a scale measuring their perceptions about how much they 
changed ir. certain areas as a result of experiences with various 
'change agents" (persons, events, facilities, or organizations) 
associated with the institution. 

Highest degree or certificate planned: Number and percentage of 
students and/or former students identifying a certain degree or 
certificate as the highest planned. 

Students enrolled in an organized educational activity for no 
credit: Number and percentage of students enrolled in organized 
educational activities for no credit within a certain period of 
time. 

Program completers curing a certain time period: The number and 
percentage of students completing a degree or certificate during a 
certain period of time; by student program. 

Program competers who entered as transfer students: Number and 
percentage of students who entered as transfer students earning a 
degree or certificate during a certain period of time by status at 
entrance. 

Degrees and certificates earned by an entering class of students: 
Number and percentage of students in a designated entering class 
who have earned a degree or certificate from the institution 
within a certain period of time, by type of degree or certificate, 
student status at entrance, and student program (field of study). 

Time to program completion for a graduating class: Amount of time 
it takes a student in a particular graduating class to earn a 
degree or certificate, by degree or certificate type, student 
major program, and student status at entrance. 

Time to program completion for an entering class: Amount of time 
It takes a student in a particular entering class to earn a degree 
or certificate, by degree or certificate type, student major 
program, and student status at entrance. 
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260 Educational program dropouts: The number and percentage of 
tuii-time students in degree or certificate programs who left the 
institution prior to completion of their program, by student level 
and exit status. 

261 Students seeking additional degrees and certificates: Number and 
percentage of exiting or former students who have been admitted or 
are seeking admission to another educational program which when 
completed will result in a degree or certificate, by type of 
degree or certificate and by student major program. 

262 Students working toward and receiving another degree or 
certificate: Number and percentage of exiting or former students 
who are worKing toward or have received another degree or 
certificate, by degree/diploma/certificate type and by student 
major program. 

263 Student ability to transfer credits: Number and percentage of 
exiting and/or former students who have successfully transferred 
credits to another school. 

264 Level of achievement of former students in another institution: 
Number and percentage of former students achieving a certain grade 
point average in another institution. 

265 Student satisfaction with overall educational experience: The 
responses of students to questionnaire items measuring the degre° 
ot satisfaction with their overall college education experience. 

266 Student satisfaction with vocational preparation: The responses 
Of students to questionnaire items measuring the degree of 
satisfaction with their vocational preparation. 



267 Student 
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bdtistaction with knowledge and skills in the humanities 
area: The responses of students to questionnaire items measuring 
the degree of satisfaction with their knowledge and skills in the 
humanities, including philosophy, literature, the arts, and 
language. 

Student satisfaction with critical thinking ability: Th- 
responses of students to questionnaire items measuring the degree 
of satisfaction with their ability to formulate ard analyze 
problems. ■' 

269 Student satisfaction with human relations skills: The responses 
ot students to questionnaire items measuring the degree of 
satisfaction with their progress in achieving human relations 
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270 Student success in obtaining first job: Number and percentage of 
students (graduates and nongraduates ) who are employed within a 
certain time period after leaving the institution. 

271 Student success in obtaining preferred first 30b: Number and 
percentage of students who received the job of their first choice 
upon leaving the institution. 

272 Occupational Career Choice: Number and percentage of students 
choosing a particular occupational career (that is, their 
employment goals). 

273 Job Satisfaction: The general satisfaction of former students 
with their job experiences. 

274 First 30b earnings: Level of earnings of exiting and former 
students on their first full-time job (35 hours or mote a week) 
after leaving school. 

275 Annual tocal income of former students: Number and percentage of 
former students who are at a particular annual income level within 
a certain time period after leaving the institution. 

276 Employment in major field of study: Number an^ percentage of 
exiting or former students who are employed in a job related to 
their program of study. 

277 Change and Stability of Career Goals: The number and percentage 
of former students who have maintained or changed their career 
goals between the time they left the institution and the present 
time. 

278 Enrollment of non-degrcs and non-certif icate seeking students: 

The number of persons, who are not seeking a degree or certificate 
(nonmatriculating students), enrolled in regular credit-producing 
instructional programs or courses, as defined by Subprograms 1.1 
and 1.2 in the NCHEMS Program Classification Structure (PCS). 

279 Community participation in community education programs: The 
number, of persons, who are not seeking a degree or certificate 
(non-matriculating students), enrolled in non-credit-producing 
instructional activities that are offered on or off campus. 
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Community participation in extension services: The number of 
persons from the community who have participated in cooperative 
extension service activities as defined by Subprogram 3. J in the 
NCHEMS Program Cl'ssif ication Structure (PCS). 



281 Educational goals achieved by community participants: The degree 
of perceived personal improvement and satisfaction with respect to 
job promotion and salary increase, development of technical 
skills, leadership and human relations, and other personal 
attributes among community participants in institutional programs. 

282 Institution's participation in community affairs: The number of 
faculty, staff, and students who participate in various types of 
off -campus activities in the community, such as workshops, 
consulting, or giving lectures. 

283 Community participation in an institution's social, cultural, and 
recreational programs: The number of persons from the community 
who participate in social, cultural, and recreational activities 
organized and sponsored by an institution for its members and the 
general public during a specified period of time. 

284 Community use of institutional facilities The number of persons 
from the community utilizing facilities maintained by the 
institution such as libraries, language labs, testing centers, 
computer centers, health services, recreation and athletic 
facilities, museums, and so forth. 



285 Institution's payment of local and state taxes and tax 
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-86 Institution s purchase of locally provided utilities: Total amount 
of dollars expended on utilities (such as qas , electricity, 
garbage collection, sewage treatr-Pnt) which were purchased from 
tne local communitv during a certain time period. 

287 Institution's purchase of locally delivered goods and services: 
Total amount of dollars expended on goods and services that are 
purchased by the institution from the local community during a 
certain time period. Goods and services are distinguished from 
capital equipments generally defined by each institution in terms 
of dollars and duration (see COMMENTS). Also, goods and services, 
as referred to here, do not include utilities purchased from the 
local community (see Outcome Measure K-2). 

Institution's capital equipment expenditure relevant to the local 
community: Total amount of dollars expended in the local 
community by institutions as a result of an institution's capital 

Capital outlay is usually defined in terms of 
a good with the cost exceeding (1) a certain amount of dollars 
and (2) the duration of useful life of the "good" years. The 
criteria may vary somewhat among institutions. (See COMMENTS) 
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289 Institution s capital construction expenditure relevant to the 
local community: Total amount of dollars expended in the local 
community by an institution as a result of its capital 
construction expenditure. The capital expenditures include (1) 
purchase of land, (2) land improvement, (3) construction (building 
and parking lot), (4) building repair and improvement, (5) 
architect s fees, and (6) others that are specifically designated 
by each institution as "capital construction." 

i'JSf^r^!"'^^^'''^^ faculty and staff: Total amount of dollars 
that the faculty and staff in an institution spend in the local 
community during a certain period of time. 

291 Local expenditures by students: Total amount of dollars that 
students spend in the local community during a certain period of 
t 1 me . ^ 

292 Local expenditures by visitors: A total amount of dollars that 
visitors to an institution spend in the local community during a 
certain period of time. ^ 

293 Yield ratio— percentage of accepted students who actually enroll. 

294 Ability levels of entering students-e.g . , as measured by SAT scores, 
high school gpa, number of exceptions to admissions standards. 

295 Dependency on community college transfers. 

296 Dependency on financial aid. 

297 Program preferences of high school seniors. 

298 Percent full-time faculty. 

299 Percent tenured faculty. 
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Distribution of faculty by rank. 



301 Faculty age distribution. 

302 Faculty and program trends— the distribution of full-time faculty by 
tenured*^ fields, and the percent of those in each field who are 

303 Faculty salaries. 

304 Faculty with terminal degrees. 

305 Graduate/professional school rate of attendance. 

306 Patterns of expenditures-trends in the patterns of expenditures in 
^r^?yo..E^o'^""'^ ""^^ indicate developing problems in terms of ability 
to allocate resources to high/low changing priorities. 
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307 Expenditures by major program area— changes in the percentage of 
educational budget may indicate areas where mandatory or fixed 
expenditures are diluting resources in other areas. 



308 Salaries as a percent of educational expenditures— may be useful for 
Identifying future problems. For example, an increasing percentage of 
budget devoted to salaries may indicate that inadequate funds are 
being devoted to overhead objects such as supplies, equipment, 
travel, etc. Conversely, a decreasing percentage may jeopardize the 
ability of the institution to hire sufficient personnel, and/or 
compensate them adequately. 

309 Energy costs as a percent of educational expenditures. 

310 Backlog of deferred mainteiiance . 

311 Plant utilization— commonly expressed as the ratio between net 
assignable square feet and full-time equivalent day students. 

312 Occupancy rate of dormitories— ref lects on abilitv to service fixed 
debts and operation costs. 

313 State tuition funding. 

314 Private funding. 

315 Short-term unrestricted current fund ratio— ratio of unrestricted 
current fund assets to unrestricted current fund liabilities. 

316 Intermediate term available funds ratio— ratio of unrestricted 
current fund balance plus quasi-endowment market value to educational 
and general expenditures plus mandatory transfers. 

317 Long-term endowment ratio— ratio of endowment market value to 
educational and general expenditures plus mandatory transfers. 

318 Rate of return on endowment--return on investment as a percentage of 
market value. 

319 Payout rate on endowment— earnings used as a percentage of market 
value at beginning of year. 

320 Gifts rate— gifts to endowment as a percentage of market value at 
beginning of year. 

321 Growth rate of expenditures— percentage increase in educational and 
general expenditures plus mandatory transfers. 

322 Debt service to revenue ratio— this ratio measures the flexibility of 
the institution to commit revenues to resources rather than debt 
service. 
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323 Acceptance rate— ratio of acceptances of freshman and transfer 
applicants to applications. 

324 Tenured faculty ratio— ratio of tenured faculty with long-term 
contracts to FTE faculty. 

325 Rejection rates— proportion of applicants actually rejected by the 
college. ^ 

326 Average test scores of entering fieshmen— e .g . , on SAT, high school 
GPA, etc. 

327 Instruction proportion— ratio of instructional expenditures to 
educational and general expenditures plus mandatory transfers. 

i"^^'^"^^^^" P^'^ ^TE student — ratio of instructional expenditures to 
FTE students. 

329 Tuition and fee rate per year. A measure of selectivity. 

330 Degree and extent to which enterprise operations are spelled out in 
plans (i.e., preprogrammed). 

331 Methodologies, techniques, and tools used in planning and decision 
making . 

332 Employee participation in the planning process. 

333 Degree and extent of information distortion in the planning process. 

334 Nature, extent, and rate of innovation and risk taking in enterprise 
operations. 

335 Types of strategic performance and control standards used in 
different areas; e.g., production, marketing, finance, personnel, 
etc . 

336 Types of control techniques used. 

337 Decree of centralization or decentralization of authority. 

338 Degree of work specialization (division of labor). 

339 Spans of control. 

340 Basic departmentation and grouping of activities. 

341 Extent and use of staff generalists and specialists. 

342 Extent and degree of organizational confusion and friction regarding 
authority and responsibility relationships. 
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343 Methods used in recruiting personnel. 

344 Criteria used in selecting and promoting personnel. 

345 Techniques and criteria used in appraising personnel. 

346 Nature and uses of job descriptions. 

347 Nature, extent, and time absorbed in enterprise training programs and 
activities. r> t- 

348 Levels of compensation. 

349 Policies and procedures regarding the layoff and dismissal of 
personnel . 

350 Ease or difficulty in dismissing personnel no longer required or 
desired. 

351 Degree and extent of authoritarian versus participative management. 

352 Techniques and methods used for motivating managerial personnel. 

353 Techniques and methods used for motivating nonmanagerial personnel. 

354 Degree and extent of identification that exists between the interests 
and objectives of individuals, work groups, departments, and the 
enterprise as a whole. 

355 Degree and extenr if trust and cooperation or conflict and distrust 
among personnel of all types. 

356 Degree and extent of frustration, absenteeism, and turno/er among 
personnel. 

357 Degree and extent of wasteful time and effort, resulting from 
restrictive work practices, unproductive bargaining, conflicts, etc. 

358 Time horizons for plans and planning. 

359 Articulation of employee development opportunities. 

360 Degree to which the leaders reflect the goals of the organization in 
their own behavior. 

361 Shifts in the socio/economic and ethnic mix of the population of the 
institution's service area. 

362 Changes and fluctuations in labor demand for business, industry, and 
government located in the college service area. 

363 Index of enterprises targeted for subsidies by the federal 
government. 
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364 Live births and the demand for teachers. 

365 Source of students by geographic area. 

366 Occupational trends. 

367 Changing student profiles. 

368 Student tlow from high schools. 

369 Trends of student interest in disciplines and professions. 

370 Manpower markets. 

371 Four-year college offerii,^ two-year programs. 

372 Average student loads. 

373 Amount of uncertainty in obtaining the next budget. 

374 Physiczil environment of the campus. 

375 Proportion of total operations budget composed of soft money. 

376 Decrease in the number of transfer students from the community 
col leges. 

377 Admission Standards. 

378 Hiring from within. 

379 Unmet program needs. 

380 Program mix and potential clientele. 

381 Increasing unit costs. 

382 Increasing percentage of faculty are part-time. 

383 Percent of faculty teaching outside of their primary fields of 
specialization. 

384 Regular faculty assigned to unusual teaching hours. 

385 Institution not staffing at traditional constant dollar rate for 
additional students . 

386 Encouragement of early retirement of faculty and staff. 

387 Proportion of faculty with over- or underloads. 

388 Period between closing date for applicetion and actual registration 
data • 
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389 Drop in application rates for admission. 

390 Increasing or decreasing dropout rates. 

391 Placement of graduates. 

392 Supply, equipment, and travel budgets. 

393 Faculty salaries growing disproportionate to the total budget. 

394 Application of, or increased fees for sr-ioport of selective services. 

395 Inr.t- tutional accreditation. 
395 Program accreditation. 

397 Educational v-rogram content— incluoing required major courses, 
required other courses, elective courses, program adndssions 
requirements. s-.;dent credit hours, student contact hours, and 
completion require.nents . 

398 Description of faculty and staff-including race, sex, appointment 
status, type of appointment, type of position, rank-titles, tenure 
status, educational credentials, and professional certification. 

399 Description of facilities and equipment— including sizes, age, 

^f^??f^^"^''^^i ownership, condition, uses (e..g, classroom, office 
facility, study facility, general use, etc.). 

400 Description of collections-including physical units, format (e.g., 
print, mic-oform, manuscript, audiovisual, cartograph-' c , graphic, 
audio, motion picture, etc.,, acquisitions , distributions, etc. 

401 Description of revenues— including cash, investments, notes 
ivJf^^^ , ' ""^^'^^wn appropriations, inventories, as well as other 
HEGIS-related revenues. 
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^!!''Ki^^^5".°^ liabilities-including notes, bonds, and mortgages 
payable, deferred revenues, owner equity, as well as other HEG::S- 
related liabilities. 



f students-including number, race, sex, age, enrollment 
status, level, geographic origin, educational credentials, 
objectives, aptitude, etc. 



404 

etc . 



Description of demand for instructional services as measured by 
student credit hours, student contact hours, FTE course enrollment, 
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405 Measures commonly employed to assess educational climate include 

Stern's (1970) College Characteristics Index , Pace's (1968) College 

and Univ ersity Environment Scales , Astin and Holland's (196lF^ 

Enviroi. mental As sessment Technique , and Halpin and Croft's (1962) 
Organiza tional Climate Description Questionnaire . Measures of 
specific constructs may be found in the BRS-ETSF and MMYD databases. 
Descriptions of these databases are reported in Appe.idix 6. 

Measures commonly employed to assess organizational climate mr^ude 
Hemphill and Westie's Group Dimensionc Descriptive Questionnaire, 
Taylor and Bower's (1972) Survey of Organizations, Stern's (1970) 
Organizational Climate Index, Litwin and Stringer's (1968) Climate 
Questionnaire, House and Rizzo's (1972) Organizational Climate 
Questionnaire. Measures of specific constructs may be found in the 
BRS-ETSF and MMYD databases. Descriptions of these databases are 
reported in Appendix 6. 
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Financial independence — dispersion of revenue sources. 



408 Revenue drawing power—ability of an institution to attract revenues 
relative to other similar institutions. 

409 Revenue stability—variability of sources of revenue over time. 
Average standard deviation of each revenue source over time. 

410 Reserve strength— average of the net increases or decreases for a 
given year across fund group 
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Criteria Measured By Ea'^h Indicator 
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4315 


4318 


4700 


329 


5 


4700 


4316 




330 


6 


2100 






331 


6 


2100 






332 


6 


2100 






333 


6 


2100 






334 


6 


2100 






335 


6 


2400 






336 


6 


2400 






337 


6 


2200 






338 


6 


2200 






339 


6 


2200 






340 


6 


2200 







Indicator 



ID# 


Ref 


a 


b 


341 


6 


2200 




342 


6 


2200 




343 


6 


2500 




344 


6 


2500 




345 


6 


2500 




346 


6 


•2500 




347 


6 


2300 




348 


6 


2500 




349 


6 


2500 




350 


6 


2500 




351 


6 


2300 


3700 


352 


6 


2300 


3500 


353 


6 


2300 


3500 


354 


6 


2300 


3400 


355 


6 


2300 


3600 


356 


6 


2300 




357 


6 


2300 




358 


6 


2100 




359 


6 


2300 




360 


6 


2300 




361 


7 


4700 




362 


7 


4700 




363 


7 


4700 




364 


7 


4700 




365 


7 


4700 


4315 


366 


7 


4700 




367 


7 


4700 


4315 


368 


7 


4700 




369 


7 


4700 


4315 


370 


7 


4700 




371 


7 


4700 


4600 


372 


7 


4315 


4316 


373 


7 


4316 


4600 4700 


374 


7 


4311 


4320 


375 


7 


4316 


4700 


376 


7 


4315 




377 


7 


4315 


4700 


378 


7 


4314 


4316 


379 


7 


4318 


4700 


380 


7 


4400 


4700 


381 


7 


4316 


4700 


382 


7 


4316 


4700 


383 


7 


4318 




384 


7 


4316 


4700 


385 


7 


4316 


4700 


386 


7 


4316 


4700 


387 


7 


4318 


4700 


388 


7 


4700 




389 


7 


'700 




390 


7 


4400 


4318 



iERlC 



155 



I 



Indicator 






ID# 


Ref 


a 


b 


c 


391 


7 


4400 


4318 


4200 


^ f\ ^ 
392 


7 


4316 


4700 




393 


7 


4316 


4700 




394 


7 


4316 


4700 




395 


8 


4318 






396 


8 


4318 






397 


8 


4318 






398 


8 


4314 






399 


8 


4311 






400 


8 


4317 






401 


8 


4316 






402 


8 


4316 






4U J 


8 


4315 






404 


9 


3800 






405 


9 


3100 


3200 


3300 


406 


10 


4316 


4700 


4600 


407 


10 


4316 


4700 


4600 


408 


10 


4316 


4700 


4600 


409 


10 


4316 


4700 


4600 


410 


10 


4316 


4700 


4600 



I 
I 
I 
I 
I 
I 
I 
I 
I 

|(2) 

l<3> 
I 
I 

|(6) 

lERiC 
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Sources of Measures and Indicators 



Lenning, Oscar r. , Lee Yong S., Micek, Service, Allan L= A Structure 
for t he Outcorres o f Postsecondary Education, Boulder, Colo,: National 
Center for Higher Education Management Systems, 1977. 

Wing, Paul, McLaughlin, James N., Allman, Katharine A. Statewide 
Measures Inventory . Boulder, Colo.: National Center for Higher 
Education Management Systems, 1975. 

Micek, Sidney S. , Service, Allan L., Lee, Yong 3. Outcome Measures and 
Procedures Manual . Boulder, Colo.: National Center for Higher 
Education Management Systems, 1975. 

Indicators of Campus Viability: A Report to the Chairman of the Senate 
Budget and Taxation Committee and House Appropriations Committee . 16 
Francis Street, Anapolis, Maryland, 21401: Maryland State Board for 
Higher Education, December, 1982. 

Dickmeyer, Nathan S. , and Hughes, K. Scott. Financial Self-Assessment: 
A Workbook for Colleges . Washington, D.C.: National Association of 
College and Univ^-r^ity Business Officers, 1980. 

Koontz, H., and O'Donnel, C. Principles of Management: An Analysis of 
Managerial Functions . San Francisco: McGraw-Hill Book Company, 1972. 
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(7) 



ERIC 



Glenny, Lyman A., and Bowen, Frank M. Signals for Chan ge; Stress 
jndicators for Colleg es and Universities . A Report to the Californi; 

Postsecond ary Education Commission . Sacramento: California 

Postsecodary Education Commission, 1981. 



(8) Christal, Melodie E. 

Postsecondai 

Collection: Boulder, Colo.: NationalCenter for"Higher EdGcation 
Management Systems, 1985. 



lelodie E., and Jones, Dennis P. A Common Language for 
try Accreditation: Categories and Definitions for Data 



(9) Campbell, John P., Bownas, David A., Peterson, Norman G., and 
Dunnette, Marvin D. The Measurement of Organizational 
Effectiveness: A Rev iew of Relevant Research and Opinio n. — San Diego, 
Calif.: Navy Personnel Research and Development Center, 1974. Copies 
may also be obtained from the National Technical Information Service, 
Springfield, Va., 22161: Document AD-786 462. 

(10) Collier, Douglas, and Patrick, Cathleen. A Multivariate Approach to 
t h e Analysis of Institutional Financial Condition . Boulder. Colo.-. 
National Center for Higher Education Managment Systems, 1978. 
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Appendix 6 
BRS Databases 



The BRS AidPagt 
for 

Mental Measurements Yearbook 



BRS Ubel: 

MMYD 

Scope: 

MMYD contains factual infor- 
mation, critical reviews and 
reliability-validity information 
on all 1184 English language 
tests covered in the Eighth 
Mental Measurements Year- 
book, plus the same content 
for very recent tests published 
after the Eighth MMY and in- 
cluded as regular updates to 
the database. 



Producer: 

Buros Institute of Mental 

Measurements 
University of Nebraska - 

Lincoln 
135 Bancroft Hall 
Lincoln, NB 68588 

Contact: 

Dr. fames Mitchell 
402-472-1739 

BIBL Paragraphs: 

AN, TN, AU, PB, PD 



lilts 



Years of Coverage: 

1977 to date, with selected 
retrospective coverage. 

Total Size: 
1186 records 

Updates: 

Monthly 

Print Counterpart: 

Eighth MentoJ Measurements 
Yearbook (and Yearbooks to 
follow) 

Royalties: 

$30 per connect hour; 

$.75 per item printed offline. 



RECORD STRUCTURE KEY 



Ubcl 


Pirigriph 


1 Function 


\ Example 


A\< 


Accession Number 


..s 


0605*175.an. 


OC 


Occurrence Table 


displa^ 


n/a 


TN 


Test Name 


..s 


strong adj 








Campbell. tn. 


CL 


Classification 


..i 


speech adj 








hearing.cl. 


AU 


Author/s 


..s 


naslund-rS.au. 


PB 


Publisher 


..i 


acif nee adj 








research adj 








associates.pb. 


PD 


Publication Dates 


..s 


1976.pd. 








•pd >* 76 






..I 


..1/1 pd < 75 


PO 


Population 




iradeSl with ' 








( 9' or •lO.po. 


LE 


Levels 


display 


n/a 



Ubtl 


Paragraph 


Function 


^ Example 


SC 


Scores 


..8 


music ad) 
theor\ .SC 


AD 


Administration 


..S 


group. ad 


MA 


Manual Information 


..s 


handbook ma 


RV 


Reliability and 
Validity Data 


..s 


8ubscores.r\ 


PR 


Price 


display 


n/a 


Tl 


Time Requirements 


displav 


n/a 


RE 


Reviewers 


..s 


nichols-rS re 


CM 


Comments 


..s 


norms with 
means. cm. 


TX 


Text 


..8 


ayres ad) space 
adj test.tx 


RR 


Reviews 


displa> 


n/a 



lERlC 
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Sample BRS/MMYD Citations 



AN' 0911-218S 

OC PARAGRAPH SENTENCE NS-WORO 

AN (1) 2 2 

AN (1) 4 2 

TN THE LOaiPOP TEST A DIAGNOSTIC SCREENING TEST 

OP SCHOOL READINESS 
CL READING 
AU CHEW ALEX L 

FB HUMAMCS LIMITED. 1182 WEST PEACHTREE STREET. 

P.O. BOX 7447. ATLANTA. CA 30309 
PD 1961. 

PD FIRST GRADE ENTRANTS 

SC 4 IDENTIFICATION OF COLORS AND SHAPES AND COPY- 
ING SHAPES. PICTURE DESCRIPTION AND POSITION AND 
SPATIAL RECOGNITION. IDENTIFICATION OF NUMBERS 
AND COUNTING. IDENTIFICATION OF LETTERS AND 
WRITING 

AD INDIVIDUAL 

MA IMl. 14 PAGES 

PR lBt3 PRICE DATA 6S PER STUDENT TEST BOOKLET. 
$)6.9S PER SPECIMEN SET OF S STUDENT TEST 
BOOKLETS. 1 MANUAL AND 1 SET OF STIMULUS CARDS 
il4.9S PER MANUAL 

n (IS) MINUTES 

RE NORRIS. lANET lUNIVERSm OF NEBRASKA-LINCOLN) 
BECK. ISABEL L. {UNIVERSITY OF PITTSBURGH) 

CM CRITERION-REFERENCED. NO NORMS. PUBLISHER 
RECOMMENDS USE OF LOCAL NORMS 

TX 1 OF 1 

THE LOLUPOP TEST A DIAGN'OSTIC SCREENING TEST 
OF SCHOOL READINESS FIRST GRADE ENTRANTS. lOei. 
CRITERION REFERENCED. 4 SCORES IDENTIHCATION OK 
COLORS AND SHAPES AND COPYING SHAPES. PICTURE 
DESCRIPTION AND POSITION AND SPATIAL RECOCNI 
TION IDFNTIFICATION OF NUMBERS AND COUNTING 
IDENTIFICATION OF UTTERS AND WRITING. NO 
NORMS. PUBUSHER RECOMMENDS USE OF LOCAL 
' ORMS. INDIVIDUAL. MA.NUAL IH 14 PAGES|. 
S'j IMULUS CARD BOOKLET TBI. 7 CARDS|. STUDENT 
TEST BOOKLET ('81. 8 PAGES). 1M3 PRICE DATA 85 PER 
STUDENT TEST BOOKLET: $19 9S PER SPECIMEN SET OF 
5 STUDENT TEST BOOKLFTS. 1 MANUAL AND 1 SET OF 
STIMULUS CARDS. S14 95 PER MANUAL (15) MINUTES. 
ALEX L CHEW. HUMANICS UMITED V 
RR 1 OF 16 

ISABEL L flECK. PROFESSOR OF EDUCATION. UNIT CO- 
DIRECTOR. READING AND COMPREHENSION UNIT 
LEARNING RESEARCH AND DEVELOPMENT CENTER. 
UNIVERSITY OF PITTSBURGH. PITTSBURGH. PA 
2 OF 16 

THE LOLLIPOP TEST HAS A .86 CORRELATION WITH 
THE METROPOUTAN READINESS TEST |MRT) THIS IS 
IMPORTANT BECAUSE THE STATED PURPOSE FOR THE 
DEVELOPMENT OF THE LOLUPOP TEST WAS TO DETER- 
MINE IF A DIAGNOSTIC SCREENING TEST OF SCHOOL 
READINESS COULD HAVE CONCURRENT VAUDITY WITH 
A WIDELY USED TEST OF PRO\'EN PREDICTIVE 
VAUDITY. . 



7 OF 16 

IN SUMMARY. THEN. THERE IS NOTHING TO RECOM- 
MEND THE LOLUPOP TEST IN TERMS OF SPECIFYING 
DISTINCT DEFICIT AREAS ANY MORE THAN THE MRT OR 
OTHER SUCH INSTRUMENTS. IT CAN BE RECOMMENDED 
AS A TEST OF PREDICTIVE VAUDITY . IF ONE WANTS A 
SHORT. EASILY ADMINISTERED. INDIVIDUAUZED IN- 
STRUMENT. FOR TYPICAL CLASSROOM SITUATIONS ONE 
MIGHT BETTER USE THE MRT OR MURPHY DURRELL 



AN 0815-1052 

OC PARAGRAPH SENTENCE NSWORD 

AN (1) 2 2 

AN (1) 4 2 

TN MINNESOTA SATISFACTION QUESTIONNAIRE 
CL VOCATIONS 

AU WEISS. DAVID ).. DAWIS RENE V.. ENGLAND. GEORGE W 

LOFQUIST LLOYD H 
PB VOCATIONAL PSYCHOLOGY RESEARCH. N620 ELLIOT 

HALL UNIVER:>ITY OF MINNESOTA 75 EAST RIVER ROAD 

MINNEAPOUS. MN 55455 
PD 1963-67 

PO BUSINESS AND INDUSTRY 

SC 3 OR 21 SCORES LONG FORM ABIUTY LTIUZATIOS 

ACHIEVEMENT ACTIVITY, ADVANCEMENT, AUTHORITY 
COMPANY POUCIES AND PRACTICES COMPENSATION 
COWORKERS CREATIVITY . INDEPENDENCE. MORAL 
VALUES. RECOGNITION. RESPONSIBILITY. SECURITY. 
SOCIAL SERVICE. SOCIAL STATUS SUPERMSION-HUMAN 
REUTIONS. SUPERVISION-TECHNICAL VARIETY WORK 
ING CONDITIONS. GENERAL SATISFACTION. SHORT 
FORM INTRINSIC EXTRINSIC GENERAL 

AD CROUP 

MA 1967. 130 PAGES 

RV NO REUABIUTY AND VAUDITY DATA FOR THE IOC- 
REVISION OF THE LONG FORM 

PR 1978 PRICE DATA 10-20 PER TEST (MINIMUM 30-15 

RESPECTIVELY), MANUAL FREE ON REQUEST SCORING 
SERVICE AVAIUBLE 

TI (15-20) MINUTES FOR LONG FORM (S-10) MINUTES FOR 
SHORT FORM 

RE CUION. ROBERT M (BOWUNG GREEN STATE 
UNIVERSITY) 

CM JOB SATISFACTION. NO NORMS FOR THE 1967 REVISION 

OF THE LONG FORM 
TX 1 OF 12 

MINNESOTA SATISFACTION QUESTIONNAIRE BUSINF^S 
AND INDUSTRY 1963-67. MSQ \Oh SATISFAaiON. 2 
FORMS. MANUAL ( 67. IX PAGES). POSTAGE EXTRA 
MANUAL FREE ON REQL^EST. DAVID I WEISS. RENE V 
DAWIS. GEORGE W ENGLAND. AND LLOYD H LOFQUIST 
VOCATIONAL PSYCHOLOGY RESEARCH -5 A) LONG FOR.M 
21 SCORES ABIUTY UTIUZATION, ACHIEVEME.NT. AC- 
TIV1TY. ADVANCEMENT. AUTHORITY . COMPAN^Y 
POUCIES AND PRACTICES. COMPENSATION. COWORKERS 
CREATIVITY. INDEPENDENCE. MORAL VALUES. RECOGNI 
TION. RESPONSIBILITY. SECURITY. SOCIAL SERVICE. 
SOCIAL STATUS SUPERVISIOVHUMAN REUTIONS. 
SUPERVISIONTECHNICAL VARIETY. WORKING CONDI 
TIONS. GENERAL SATISFACTION. NO REUABIUTY' AND 
VAUDITY DATA OR NORMS FOR THE 1967 REVISION 2 
EDITIONS ORIGINAL EDITION ('63. 7 PAGES). 1967 REVI- 
SION r87. 7 PAGES. IDE.NTICAL WITH ORIGINAL EDITION 
EXCEPT FOR RESPONSE OPTIONS. FOR RESEARCH USE 
ONLn 20 CENTS PER TEST (MINIMUM 15). SCORING SER 
VICE. 55 CE.NTS PER TEST. (15-20) MINUTES B) SHORT 
FORM 3 SCORES. INTRINSir EXTRINSIC GENERAL 1 
FORM ('63. 4 PAGES). 10 CEN i PER TEST (MINIMU.M 30) 
SCORING SERVICE. 35 CENTS PER TEST. (5-10) MINUTES 



TX 12 OF 12 

IN SUMMARY. THE MSQ IS WELL DEVELOPED. IT HOLDS 
UP WELL IN COMPARISON WITH A MAjOR ALTERNATE 
INSTRUME.NT. AND IT CAN GIVE DETAILED DIAGNOSTICS 
OR PARSIMONIOUS SUMMARY STATEMENTS ACCORDING 
TO AN IN\'ESTIGATOR S NEEDS THE ONLY REQUEST THE 
REVIEWER CAN MAKE IS THAT THE NEXT MANUAL BE 
MORE COMPLETE IN DESCRIBING THE REASONING AND 
THE HISTORY BASIC TO IT. 
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The BRS AidP^ge 
for the 

Educational Testing Service Test Collection 



BRS Ubel: 

ETSF 

Scope: 

Descriptions, abstracts and 
availability information on tests, 
evaluation tools and assessment/ 
acreening devices for measuring 
skills, aptitude, interests, 
attitudes or achievement. 

Search Aid: 

Thesaurus of ERIC Descriptors 

Updates: 

Quarterly 



Producer: 

Educational Testing Service 
Test Collection 
Princeton, N] 08541 

Contact: 

Marilyn Halpern 
609-734-5737 

Document Delivery: 

Availability information is 
included with each record 

BIBL Faragraphs: 

AN,T1,AU.AV,YR 



Years of Coverage: 

Both current and older 
assessment devices 

Total Size: 

Approximately 5000 records as 
of November 1983 

Connect Hour Royalty: 

$25 

Online Per Qtation Charges: 

Free Paragraphs: AN,DT,AU.YR, 
DE.SW,RTID,GL.TG.NT 
$.53: T1.AV,AB 

Offline Fer Qtation 
Charges: 

Free Paragraphs: AN.DTAU,YR. 
DE.SW.RTID^GL.TG.NT 
$.50: TI,AV,AB 
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RECORD STRUCTURE KEY 



Ubel 


^ragraph 


Function 


Example 




Ubel 


hragraph 


Function 


Example 


AN 


Accession Number 




tc0n945.an 




SW 


Single Word 


..s 


numbers sw 






Cg 


a309.an 






Descriptors 








^tc - 011945 










UP 


Update Code 


..s 


a309 up. 




RT 


Resource Type 


..s 


assessment rt 






Ca 
..1 


^up « a309 
.1/1 up « a309 




ID 


Identifiers 


..s 


purdue with 














inventor) id 


Tl 


Title 




pre adj algebra.ti. 




AV 


Availability 


..s 


western ad) 


DT 


Subtests 


..s 


anxiety same 










psychological. 8 V 








aggretsion.dt. 




GL 


Grade Level 


..s 


(kr-2)gl 


AU 


Author 


..% 


harrit-david-p.au 

harris.au. 
educational adj 




TG 


Target Audience 


..s 


(•16- 17 Hg 
adults tg 








tMling ie'vice.au. 




NT 


Notes 


..s 


iowa with testS nt 


YR 


Year 


..s 
Ca 


SO.yr 
&yr > 79 




AB 


Abstract 


..s 


visualh ad| 
impaired ab 


DE 


Descriptors 




speech-ikills.de 
speech ad} 
ikills.de 
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■- Sample BRS/ETSF Citations 



OKI copy AVAILABLE 



AN AOZSSION NUMBER TCOims ETS 83U9 

Tl TraE. MISSOUW OCCUPATIONAL PREFERENCE 

INVENTORY 
AU AUTHOR MOORE-EARL; AND OTHERS 
YR YEAR 80 

■E DESCRIPTORS ADOLESCENTS ADULTS CAREER. 

CHOICE. CAREER-COUNSEUNG COLLEGE STUDENTS 
HIGHER-EDUCATION HIGH-SCHOOLS HIGH-SCHOOL- 
STUDENTS INTEREST-INVENTORIES VOCATIONAL- 
INTERESTS. 

RT RESOURCE TYPE. ASSESSMENT/SCREENING 
INSTRUMENT. 

ID IDENTinERS CARD SORT. MOPI 

AV AVAILABILITY- HUMAN SYSTEMS CONSULTANTS. 1 10 
NORTH TENTH STREET. COLUMBIA. MO 65201 

GL GRADE LEVEL 9. 10. 11. 12. HIGHER EDUCATION 

TG TARGET AUDIENCE AGEl3-*4 

NT NOTES: 

TIME 60. APPROX 

AB ABSTRACT USED TO ASSIST INDIVIDUALS IN 
EXPLCMUNG CAREER OPTIONS. UNDERSTANDING 
REASONS BEHIND OCCUPATIONAL CHOICES. AND IN 
IDENTIFYING NEXT STEPS TO GUIDE FURTHER 
EXPLORATION. ADAPTABLE TO MANY COUNSEUNG 
AND CAREER EXPLORATION SITUATIONS. A VARIETY 
OF WORK SETTINGS. AND NEEDS OF VARIOUS 
POPULATIONS THERE ARE THREE FORMATS IN WHICH 
TO USE MOPI. SELF GUIDED APPROACH WHICH 
INVOLVES USING THE CARDS AS A RESOURCE RLE. 
SELF DIRECTED CARD SORT IN WHICH INDIVIDUAL 
ASSUMES RESPONSIBILITY FOR CAREER 
EXPLORATION. AND COUNSELOR DIRECTED CARD 
SORT WHICH PROVIDES FOR IN-DEPTH CAREER 
EXPLORATION AND PLANNING WITH HELP FROM A 
COUNSELOR 



AN ACCESSION NUMBER TC922001 ETS 8309 
TI TITLE COGNITIVE ORIENTATION QUESTIONNAIRE OF 
CURIOSITY 

DT SUBTESTS BEUEFS ABOUT SELF; GENERAL BEUEFS 

BEUEFS ABOUT GOALS. BEUEFS ABOUT NORMS 
AU AUTHOR KREITLER-SHULAMITH. KREITLER HANS 
YR YEAR 74 

DE DESCRIPTORS. ATTITUDE- MEASURES ATTITUDES 
BEHAVIOR COGNITIVE-STYLE CURIOSITY FOREIGN- 
COUNTRIES INDIVIDUAL-TESTING PRIMARY- 
EDUCATION SELF-CONCEPT. YOUNG^ILDREN 

RT RESOURCE TYPE: ASSESSMENT/SCREENING 
INSTRUMENT 

ID IDENTIFIERS. ISRAEL; TIM(D) 

AV AVAILABILITY. TESTS IN MICROFICHE TEST 

COLLECTION. EDUCATIONAL TESTING SERVICE. 
PRINCETON. NI 08541 

TC TARGET AUDIENCE AGE 4-8 

NT NOTES. 
ITEMS 73 

AB ABSTRACT. TEN PART QUESTIONNAIRE DESIGNED TO 
ASSESS BELIEFS THAT MAY ORIENT A CHILD TOWARD 
BEHAVIOR IDENTIFIED AS CURIOSITY. THE 73 ITEMS 
REFER TO FOUR BEUEF TYPES NORMS. GENER-\L 
BEUEFS. BEUEFS ABOUT SELF. AND GOALS 
RESPONSES ARE SCORED IN TERMS OF THREE 
CATEGORIES. PkOCURIOSITY ORIENTATION. 
ANTICURIOSITY ORIENTATION. AND INDETERMINATE 
POSITION BASIC FORM IS ALTERED TO SUIT BOYS OR 
GIRLS INDIVIDUALLY ADMINISTERED 



AN ACCESSION NUMBER TC8302e8 ETS 8309 

Tl TITLE CLARKE PARENT/CHILD RELATIONS 
QUESTIONNAIRE 

DT SUBTESTS MOTHER'S AGGRESSION TO SUBIECT 
FATHER S AGGRESSION TO SUBJECT; SUBIECT S 
AGGRESSION TO MOTHER. SUBJECT S AGGRESSION TO 
FATHER. MOTHER'S AGGRESSION TO FATHER 
FATHER S AGGRESSION TO MOTHER. MOTHER S 
COMPETENCE. FATHER'S COMPETENCE. MOTHER S 
AFFECTION. FATHER S AFFECTION; MOTHER S 
STRICn^S. FATHER'S STRICTNESS. MOTHER 
IDENTIFICATION. FATHER IDENTinCATION, MOTHER S 
INDULGENCE; FATHER'S INDULGENCE. DENIAL 
(MOTHER). DENIAL (FATHER) 

AU AUTHOR. PATTICH-DANIEL 

VR YEAR 76 

DE reSCRI PTORS. ADULTS AFFECTION AGGRESSION 
ATTTTUDE-MEASURES CHILDHOOJ ATTITUDES 
FOREIGN<X)UNTRIES INTERPERSONAL- 
KELATIONSHIP PARENTOilLD-RELAHONSHIP 
PARENT-ROLE 

RT RESOURCE TYPE: ASSESSMENT/SCREENING 
INSTRUMENT. 

ID IDENTIFIERS CANADA. PCR. TIM(D) 

AV AVAILABILTTY. TESTS IN MICROFICHE. TEST 

COL LECTIO N. EDUCATIONAL TESTING SERVICE. 
PRINCETON. N) 08541 

G TARGET AUDIENCE ADULTS 

NT NOTES 

fTEMS 131. 

AB ABSTRACT DESIGNED TO SAMPLE THE CONTENT 
AWBASOF PARENTOiILD RELATIONS THAT HAVE 
BEEN FOUND SIGNinCANT IN CUNICAL RESEARCH 
WCLUDES 18 SCALES DEAUNG WTTH AGGRESSION. 
CC ^PETENCE. AFFECTION. STRICTNESS. 
IDENTIFICATION. INDULGENCE. DENIAL 



AN ACCESSION NUMBER T0007289 ETS 8309 

Tl TTTLE lOX OBJECTIVES-BASED TEST COLLECTIONS- 
LANGUAGE ARTS MECHANICS AND USAGE. 
KINDERGARTEN^RADE 6 

DT SUBTCSTS: CAPITAU2ATI0N AND PUNCTUATION 

USING COMMAS. PLURAL NOUN FORMS. POSSESSIVE 
FORMS. PRONOUN FORMS. MODIFIERS, VERB 
AS'^HJSf' »W<ECULAR VERBS. COMMONLY 
CONFUSED WORDS 

AU INSTTTUnONAL AUTHOR INSTRUCTIONAL 
OBJECTIVES EXCHANGE. LOS ANGELES. CAI JF 

YR YEAR. 73. 

DE DESCRIPTORS CRrrERION-REFERENCED-TE.iTS 

ELEMENTARY-EDUCATION ELEMENTARY-SCHCOL- 

GRAMMAR ITEM BANKS KINDERGARTEN- 
CHILDREN LANGUAGE-SKILLS. SPEECH-SKILLS 
WRJTING-SKIJ LS 

RT RESOURCE TYPE. ASSESSMENT/SCREENING 
INSTRUMENT. 

ID IDENTIFIERS INSTRUCTIONAL OB|ECTI\TS 
EXCHANGE. lOX. 

AV AVAILABIUTY. INSTRUCTIONAL OBIECTINXS 

EXCHANGE; BOX 24095-M. LOS ANGELES. CA 90024 

GL GRADE LEVEL: K;l; 2; 3. 4. 5: 6 -^ ^n^uu^^ 

NT NOTES. SEE ALSO lOX MEASURABLE OBIECTIVES 
roLL£(niONSJLANGUAGE ARTS-MECHANICS AND 
USAGE. K-12 (TC006959). 

SKILLS IN MECHANICS AND 
USAGE OF STANDARD ENGUSH AREAS INCLUDE 
Si^^J^^i:'^'^'^'^' PUNCTUATION. ABBREVIATION. 
£XfS?i^Jl9^' BIBUOCRAPHIC FORM. PLURALS. 

PRONOUN REFERENCES SUBIECT-VERB 
AGREEMENT, IRREGULAR PAST PARTK:IPLES AND 
S2^9li^)' CONFUSED WORDS. FORMS A AND B ARE 
CONTENT PARALLEL EACH CONSISTS OF THIRTY- 
EIGHT TESTS ON FORTYONE SPIRTT MASTERS 



